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PROGRAM 


Electronic Imaging & the Visual Arts 


‘The Foremost European Electronic Imaging Events in the Visual Arts’ 


Forum for Users, Suppliers & Researchers 


The key aim of this Event is to provide a forum for the user, supplier and scientific research 
communities to meet and exchange experiences, ideas and plans in the wide area of Culture & 
Technology. Participants receive up to date news on new EC and international arts computing & 
telecommunications initiatives as well as on Projects in the visual arts field, in archaeology and 
history. Working Groups and new Projects are promoted. Scientific and technical demonstrations 
are presented. 


Main Topics 
e 2D —3D Digital Image Acquisition 
e Leading Edge Applications: Galleries, Libraries, Education, Archaeological Sites, Museums & 
Historical Tours 


e Mediterranean Initiatives in Technology for Cultural Heritage: 
Synergy with European & International Programmes 


e Integrated Digital Archives for Cultural Heritage and Contemporary Art 

e Management of Museums by using ICT Technology: Access, Guides, Documentation & Other 
Services 

e The Impact of New Mobile Communications on Cultural Heritage and Modern Arts Area 

e Semantic Webs 

e Human - Computer Interaction for Cultural Heritage Applications 

e Copyright Protection (Watermarking & Electronic Commerce) 

Culture and e-government 

Activities and Programmes for e-learning 

Application of Digital Terrestrial Television 

3D Developments and Applications in the Cultural Heritage Area 

Digital Theater 

Cultural Tourism & Travel Applications 

Art and Medicine 


WHO SHOULD ATTEND 


THE CULTURAL SECTOR: The Visual Arts Community including Museums, Libraries, 
Archaeological Sites, Educational Institutions, Commercial Galleries and Dealers, Auction Houses, 
Artists & Collectors 

THE HI-TECH INDUSTRY SECTOR: Multimedia Systems, Image Acquisition & Analysis, 
Data-bases, Display & Printing, ICT Industry, Telematics & Systems Manufacturing, On-line 
Information Services 

MEDIA & RELATED SECTORS: Publishing, Press, Film, Television, Photography, Printing, 
Advertising, Graphics Design, Consumer Media 

IMAGING SYSTEMS RESEARCHERS: Imaging Systems, 3-D Acquisition, Reconstruction & 
Representation Systems, Information Sciences 

TOURISM & TRAVEL SECTOR: Tourism Agencies & Operators, Travel Agencies 

THE GOVERNMENT SECTOR: Ministries of Culture and other Institutions involved in Cultural 
Heritage, Ministries of Industry, Education, Research and Science, Regional Governments 
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SPONSORS & SUPPORTERS INCLUDE: 


EUROPEAN COMMISSION, THE UNIVERSITY OF FLORENCE, 
REGIONE TOSCANA, COMUNE DI FIRENZE, 
DIPARTIMENTO DI INGEGNERIA DELL'INFORMAZIONE, 
CENTRO PER LA COMUNICAZIONE E L'INTEGRAZIONE DEI MEDIA, 
ISTITUTO DI FISICA APPLICATA “NELLO CARRARA” - C.N.R. 
CONFINDUSTRIA FIRENZE, 

GTTI, CNIT, 

HITACHI LTD., 

GRUPPO SESA, 

CABEL, 

T.T. TECNOSISTEMI, 

FRATELLI ALINARI IDEA, 

CENTRICA, 

NEUMUS, 

VIDITRUST, 

INN-3D, 

ICESP, 

UNIVERSITA’ INTERNAZIONALE DELL'ARTE, 


ENTE CASSA DI RISPARMIO DI FIRENZE 


Chairman: Vito Cappellini — Florence University 
Co-Chairman: James Hemsley — EVA Conferences International 


vito.cappelliniQunifiit ^ jrhemsley(g)hotmail.com 


Chairmen of International Conference Sessions and Workshops 


Paolo Blasi, University of Florence, Florence, Italy 
Enrico Bocci, Commissione ICT e Servizi Innovativi, Confindustria Toscana, Italy 
Bernd Breuckmann, Breuckmann 3D-Engineering, Germany 
Vito Cappellini, University of Florence, Florence, Italy 
James Hemsley, EVA Conferences International, London, U.K. 


Takayuki Morioka, DIS Project, Hitachi Ltd., Yokohama, Japan 


Konstantine Karczmarski, University Innovation Office, ITMO University, Saint Petersburg, Russia 


EVA Conferences Organiser: Monica Kaayk, acinom_S@hotmail.com 


EVA 2015 Florence Technical Organising Committee includes: 
Roberto Caldelli, Alessandro Nozzoli, Alessandro Piva, Francesca Uccheddu, 
Luca Belloni, Riccardo Saldarelli, Giuliana Signorini, 

Silvia Capecchi, Paola Imposimato, Claudia Riva di Sanseverino 


9, ENTE 
@P ©, CASSA DI RISPARMIO 


SY Di FIRENZE 


EVA Organiser: Centro per la Comunicazione e l'Integrazione dei Media (MICC) - 
Dipartimento di Ingegneria dell Informazione (DINFO) 

Universita degli Studi di Firenze 

Viale Morgagni 65 — 50134 Firenze, Italy 

Tel.: (+39) 055 2751391 - Fax: (+39) 055 2751396 

E-mail: eva2015florence@gmail.com 

Web pages — http://iapp.det.unifi.it/uploads/documents/highlights/Programme.pdf 


For general information: Prof. Vito Cappellini & Secretariat 
Centro per la Comunicazione e l'Integrazione dei Media (MICC) 
Viale Morgagni, 65 - 50134 Firenze, Italy 
Tel.: (+39) 055 2751391 — Fax: (+39) 055 2751396 
E-mail: vito.cappelliniQunifi.it 


For information on the Exhibition: CENTRICA S.r.l. 
Piazza della Madonna della Neve, 5 - 50122 Firenze - Italy 
Tel.: (+39) 055 24 66 802 — Fax: (+39) 055 20 09 785 
E-mail: info@evaflorence.it Web pages — http://www.evaflorence.it/home.php 


PROGRAM 


1- CONFERENCE 

Wednesday, 13 May: 14,15 — 19,10 
Thursday, 14 May: 9,00 — 17,55 
2-WORKSHOPS 


Wednesday, 13 May: 9,00 — 13,00 
3 - SPECIAL EVENT 
Wednesday, 13 May: 20,00 — 22,45 


4 - TECHNICAL EXHIBITION 
Thursday, 14 May: 10,00 — 17,00 


1 - CONFERENCE 


Wednesday, 13 May 


ROOMA 


Chairmen: 


Vito Cappellini, Florence University 


James Hemsley, EVA Conferences International 


14,15 Opening: Enrico Del Re, 


Director Dipartimento di Ingegneria dell'Informazione, 
University of Florence 

Pier Luigi Rossi Ferrini 
Vice-President Ente Cassa di Risparmio di Firenze 


Dirk Petrat, 


Ministry of Culture, Free and Hanseatic City of Hamburg 


Fabio Donato, 


University of Ferrara, Italian Representative in Horizon 2020 
- SC6 Committee 


15,30 Coffee Break 
15,45 SESSION 1 — STRATEGIC ISSUES 
Chairman: 


"Research Infrastructures and e-Infrastructures 
for Cultural Heritage Communities" 


“The Strategic Role of IP Rights Protection" 


"Interpreting Heritage: Where Technology Meets 
the Audience" 


"Museo Nazionale del Bargello: Mnemosyne - 
Smart Museum Project" 


"High Quality Archive Project For Polo Museale 
Fiorentino: Exploitation Activities" 


Vito Cappellini, Università di Firenze, Florence, Italy 


Rossella Caffo 
Michael-Culture Association 


Daniela Mainini 
Anti-Counterfeiting Center, Milan, Italy 


Margaretha Mazura”, Kaja Antlej? 

'EMF - the Forum of e-Excellence, Bruxelles, Belgium 
Centre for Creative and Cultural Research (CCCR), 
Faculty of Arts & Design, University of Canberra, 
Canberra, Australia 


llaria Ciseri 
Museo Nazionale del Bargello, 
Florence, Italy 


Vito Cappellini’, Takayuki Morioka?, Marco 
Cappellini? 

'MICC - University of Florence, Italy 

?DIS Project, Hitachi Ltd., Yokohama, Japan 
Centrica S.r.l., Florence, Italy 
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ROOMA 


17,50 SESSION 2 - NEW TECHNOLOGIES & APPLICATIONS 
Chairman: Konstantine Karczmarski, University Innovation Office, ITMO University, 
St.Petersburg, Russia 


"International Cluster Approach as a Fast Track Konstantine Karczmarski 
to Practical Realization of Disruptive Innovations University Innovation Office, 
in Culture" ITMO University, 

Saint Petersburg, Russia 


"The Reinvention of Print in our Digital World: Andrea De Polo Saibanti 
It's Personal, Powerful, Permanent" Fratelli Alinari IDEA Spa, 
Florence, Italy 


"ViSeQRG: a passive authentication technology” Giacomo Cancelli, Marco Cappellini 


ViDiTrust S.r.l., 
Siena, Italy 
"Patient and User: Techniques for Treatment L. Burzagli, F. Baldini, R. Pini 
and Interaction with People" Istituto di Fisica Applicata "Nello Carrara" - CNR, 


Sesto Fiorentino, Florence, Italy 


Thursday, 14 May 
ROOM A 


9,00 INTERNATIONAL FORUM ON “CULTURE 8 TECHNOLOGY 
Chairman: Paolo Blasi, Università di Firenze, Florence, Italy 


The structure of the FORUM is presented. 
Actual developments and perspectives are outlined: 


- Cooperation Groups 

- Proposed Projects 

- Funding Opportunities 

- European Commission Plans 
(HORIZON 2020) 


Speakers Include: - Alberto Del Bimbo, Director Centro per la 
Comunicazione e l'Integrazione dei Media, 
Florence, Italy 
- Cristina Acidini, ex-Superintendent of Polo 
Museale Fiorentino, Florence, Italy 
- Monica Carfagni, President Promo Design, 
Calenzano, Florence, Italy 
- Takayuki Morioka, Director DIS Project, 
Hitachi Ltd., Yokohama, Japan 
- Edoardo Calia, Research Director, Istituto 
Superiore Mario Boella, Torino, Italy 


11,00 Coffee Break 


11.15 
APPLICATIONS 


Chairman: 


"Close Range 3D Scanning in Cultural 
Heritage: Discussion of Different Technologies" 


"Collaborative Interactions on Media Facades" 


“FTV for Revolutionized 3D Viewing" 


"Free-viewpoint Image Generation from Free 
Setup Cameras" 


"Word-of-Mouth Marketing and Social Media 
for Russian Cinema-Goers" 


"Proving the Story" 


SESSION 3 - NEW 2D-3D TECHNICAL DEVELOPMENTS & 


Bernd Breuckmann Breuckmann 3D- Engineering, Meersburg, Germany 


Bernd Breuckmann 
Breuckmann 3D - Engineering, 
Meersburg, Germany 


Jürgen Sieck'?, Desiree Windisch? 

'INKA, University of Applied Sciences, Berlin, 
Germany 

?School of Computing and Informatics, Polytechnic of 
Namibia, Namibia 


Masayuki Tanimoto 
Nagoya Industrial Science Research Institute, 
Nagoya, Japan 


Masayuki Tanimoto 
Nagoya Industrial Science Research Institute, 
Nagoya, Japan 


A. Peregudov 

Center of Research and Innovation for Digital Cinema, 
Saint-Petersburg State Institute of Cinema and 
Television, 

Saint-Petersburg, Russia 


Greg Loftin 

School of Production, 
Ravensbourne, 
London, U.K. 


13,15 Lunch Break 
ROOMA 
14,40 SESSION 4 - VIRTUAL GALLERIES - MUSEUMS 


AND RELATED INITIATIVES 


Chairman: Takayuki Morioka, DIS Project, Hitachi Ltd., Yokohama, Japan 


"Enhancing Viewer's Emotional Connections 
to the Traditional Art Creative Process 
Via An Al Interactive System" 


"Methodologies and Technologies for 
Restoration. SMARTICON: Information Retrieval 
and the Rediscovery of an important Painting" 


"It's Those Einstein Pictures again! The Users 
of ETH-Bibliothek's Image Archive: An Image - 
Science Study" 


Sara Salevati, Steve Di Paola 
School of Interactive Arts & Technology, 
Simon Fraser University, 

Vancouver, Canada 


Sara Penco 
Restorer and Creator of the “SMARTICON Project”, 
Rome, Italy 


Nicole Graf, 

Image Archive, 
ETH-Bibliothek, ETH, 
Zurich, Switzerland 
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"Digital Exhibit: Cross-Media and Interactive Laura Longo’, Nicola Amico?, 

Digital Storytelling for Archaeological Heritage" Cinzia Luddi?, Ginevra Niccolucci? 
'Musei Civici Fiorentini, Florence, Italy 
?PIN, VAST-LAB, Prato, Italy 


16,00 Coffee Break 
ROOMB 
16,15 SESSION 5 - ACCESS TO THE CULTURE INFORMATION 


Chairman: | James Hemsley, EVA Conferences International, U.K . 


"Culture Clock & The e-Culture Cloud" Mathias Müller-Using 
Interpol*- Studios / Nordpol* Hamburg, 
Berlin / Hamburg, 
Germany 


“Augmented Reality to Improve Users Experience Raffaella Folgieri', Marco Granato? 

in Art: An Application of Epson Moverio and 'Dipartimento di Economia, Management e Metodi 

Google CardBoard Devices" quantitativi, Università degli Studi di Milano, Milan, Italy 
CdL Cognitive Science and Decision Making, 
Università degli Studi di Milano, Milan, Italy 


"Be Smart: Stay Human why e-Learning Needs Michael Petermann 
Real Teaching" Hamburg Conservatory, 
Hamburg, Germany 


"Transmitting Hybrid Theatre & Cinema Across James Hemsley’, James Morris? 


Borders with Live English, German & Italian 'Birkbeck College, University of London and EVA, 
Language Events for Education & Enjoyment" London, U.K. 

Ravensbourne, London, U.K. 
"Social Networks Analysis to Enhance The F. Spadoni!, F. Thétis’, R. Rossi!, F. Tariffi?, 
Cultural Experience in the NeoLuoghi Project” ‘Rigel Engineering S.r.l., Livorno, Italy 


?Space S.p.a., Prato, Italy 


2 - WORKSHOPS 


Wednesday, 13 May 
ROOM B 


WORKSHOP 1 INTERNATIONAL COOPERATION 
9,00 — 13,00 Chairman: James Hemsley, EVA Conferences International 


The general aspects of international cooperation in Cultural Heritage are presented. The 
impact of new technologies in the field is considered, outlining the more suitable ones for 
cooperative plans. 

The importance of Virtual Heritage for better cooperation among the Nations in the World is 
considered. 

Coordination and promotion Lines by International Organizations (as by UNESCO) are 


outlined. 


Speakers include: - Maria Luisa Stringa, Presidente Centro UNESCO di Firenze, Florence, 
Italy 
- Nikolay Borisov and Vera Slobodyanuk, Center of Design and Multimedia, 
ITMO State University, Saint Petersburg, Russia 
- Gioacchino Onorati, Editore ARACNE EDITRICE, Rome, Italy 
- Roberto Fracassini, Console Onorario d Italia, Guam, USA 
- Elisabetta Susani, Academia di Belle Arti, Brera and Politecnico di Milano— 
Dipartimento DAStU, Milan, Italy 
- Antonia Ida Fontana, Presidente Società Dante Alighieri, Florence, Italy 
- Marco Cappellini, Centrica S.r.l., Florence, Italy 


ROOMA 

WORKSHOP 2 INNOVATION AND ENTERPRISE - INNOVAZIONE E IMPRESA 

(Italian Language) 

9,00 — 13,00 

Chairman: Enrico Bocci, Responsabile Commissione ICT e Servizi Innovativi 
Confindustria Toscana e Presidente Opera Medicea Laurenziana, Florence, 
Italy 


Technological requirements in the Cultural Heritage field are outlined and opportunities for 
Italian SME's working in the field, using new technologies, are presented. 

Regional and national applied research Programs in Italy are described. 

Activities by National Organizations and Firms working in the area of Telecommunications, 


Informatics, Environment and Infomobility are presented. 
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Funding by European Commission is considered, with particular reference to multimedia 
and telematics for Cultural Heritage. Special consideration is given to the new EC Plan 
HORIZON 2020. 

Initiatives regarding the “know-how” transfer from Research Organizations to the Industrial 
Sector are described. 


Organizations and Companies present their activities and experiences. 


Invited Speakers: - Marco Bellandi, Pro-Rettore al Trasferimento Tecnologico 
e Presidente di CsaVRI, Università degli Studi di Firenze 
- Daniela Belliti, Vice-Sindaco, Comune di Pistoia 


Speakers include: ^ - Luigi Perissich, CONFINDUSTRIA Servizi Innovativi e Tecnologici 
- Renzo Zampini, INFOCAMERE 
- Paola Castellacci, VARGROUP 
- Daniele Corsini, CABEL 
- Riccardo Bruschi and Luca Bencini, T.T. Tecnosistemi, Prato 
- Gianluca Vannuccini, Servizio Sviluppo Infrastrutture Tecnologiche, 
Comune di Firenze 
- Andrea del Re, Studio Legale Del Re — Sandrucci, Firenze 
- Franco Guidi, NEUMUS, Firenze 
- Martina Ruffo, CON-SCIENZA, Firenze 
- Danilo De Roberto, XY Project S.r.l., Firenze 
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3- SPECIAL EVENT 


Wednesday, 13 May 20,00 — 22,45 


RECEPTION at Grand Hotel Minerva 


Multimedia Presentation of Art and Science 
(in cooperation with Antica Compagnia del Paiolo) 


4 - TECHNICAL EXHIBITION 


Thursday, 14 May: 10,00 — 17,00 


For information on the Exhibition: 

CENTRICA S.r.l. 

Piazza della Madonna della Neve, 5 - 50122 Firenze - Italy 

Tel.: (+39) 055 24 66 802 — Fax: (+39) 055 20 09 785 

E-mail: info@evaflorenceit ^ Web pages ~ http://www.evaflorence.it/home.php 


CONNECTED EVA 


EVA 2015 SAINT PETERSBURG 


24-25 June, Saint Petersburg, Russia 


Chairmen: 
Vito Cappellini Nikolay Borisov 
University of Florence Saint Petersburg Research University 
of Information Technologies Mechanics 


and Optics (ITMO) 


Reference: http://evaspb.ifmo.ru 


PROCEEDINGS 


SIRATEGIC ISSUES 


Research Infrastructures and e-Infrastructures for cultural heritage communities 
Rossella Caffo 
President of Michael-Culture Association 


rosa.caffo@beniculturali.it 


The existing e-Infrastructures for research and academia are efficient channels for the delivery of 
advanced services that can be used by the digital cultural heritage communities such as stable 
technical platforms, federated authentication and authorization infrastructure, cost-effective data 
storage and preservation and superior processing power. 


The research infrastructures in the sector of cultural heritage, humanities, history, archaeology 
and so on, are integrating the existing different research data infrastructures on cultural heritage, 
setting their own features that characterize a distinctive research methodology, so as to allow 
researchers to use distributed data and new and powerful technologies. 


Virtual research environment in cultural heritage 


The ESFRI projects and other projects focused on research in cultural heritage fields, like DARIAH, 
ARIADNE are working for designing and creating facilities, resources and related services for the 
research community: a kind of a digital ecosystem i.e. a digital environment in which members — 
i.e. researchers, research teams and research institutions — will create a virtual research environment 
providing access to all their content, services and applications that will enable exchanging expertise, 
methodologies, and practices across domains and disciplines. The challenge is to foster cooperation 
between e-Infrastructure providers and cultural heritage communities for a common vision on a 
sustainable, shared public e-Infrastructures in the cultural heritage domain. 


Need of common policies 


The impact of e-Infrastructures on virtual research and cultural heritage communities must take 
into account the regulation and governance of e-Infrastructures: there is a need to establish 
common policies, processes and protocols which will allow digital cultural heritage (DCH) 
organisations to access e-Infrastructures, despite the fact that NRENs and NGls are national 
entities, often with different policies and procedures for access and usage. 


The dialogue between e-Infrastructure providers and cultural heritage sector: projects 
coordinated by Italian Ministry of Culture 


The task of establishing a dialogue and a relationship between the cultural sector, the research 
sector and the e-infrastructures in Europe was successful undertaken by three European projects, 
conducted under the leadership of the Italian Ministry for Cultural Heritage and Activities and 
Tourism. The launch of this dialogue was firstly realized with the DC-NET project and then 
renewed with INDICATE and DCH-RP projects that investigated political and technical domains 


concerning the relation between the DCH sector and e-Infrastructures. 
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New skill and competences 


The challenges raised by the digital world on traditional professional practices are of large-scale, 
and this is particularly true in the cultural heritage sector; there is a need of new professions and skills 


for E-Infrastructure. 
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THE STRATEGIC ROLE OF IP RIGHTS PROTECTION 


Daniela Mainini 
President of Anti-Counterfeiting Center 
presidenza@anticontraffazione.org 


ABSTRACT 


Intangible rights protect the products of human intelligence, such as copyrightable works, 
patented inventions, trademarks and trade secrets. The protection and the enforcement of IPR are 
crucial for the development of the industrial sector and its ability to compete in a global economy 
- both on a national and international level. Counterfeiting should be considered a virus which 
insinuates itself into the healthy economy, damaging entrepreneurs, legal employment and 
fostering criminality. In recent years, concerns for public health and safety have increased due to 
the replication of particular kinds of goods, such as medicines, foods, beverages, toys and spare 
parts for automobiles and aircrafts. Counterfeiting should be viewed neither from a political right 
nor from a political left but from above and below: “below” means admitting that counterfeiting is 
part of an insalubrious submerged economy; "above" signifies that we need a strategic vision to 
combat it. The first priority in this battle is growing awareness of the phenomena, particularly 
among consumers, with a strong focus on young people. Our methods of communication need to 
change. We must adopt modern technology, using Social networks, such as Twitter or Facebook. 
Enforcement is then necessary: even if Italian law is fairly adequate, this remains the greatest 
issue: in Italy, specialization in Intellectual Property Rights can be found in civil court judges but is 
almost totally absent among criminal judges. Despite the creation of Italian Enterprise Courts into 
which the IP Civil Courts have been absorbed, the challenge remains the one of preserving the IP 
specialization of civil judges and of creating that of criminal Judges. The next goal is coordination: 
setting up a series of committees to roll-out anti-counterfeiting initiatives. We must stop 
cultivating "small gardens" of skills to protect IP, starting to carry out transversal investigation. 
Growth of awareness among entrepreneurs is also crucial: the protection and the enforcement of 
IPR must be recognized as necessary prerequisites for the development of Italian industry. Even in 
SMES there is a need for an IP manager, which should cover a strategic position. Internet and the 
web will certainly open new avenues for international trade, but at the same time the 
globalization will offer new possibilities to the criminals: the future battle against counterfeiting 
will definitely be conducted on the web, since it is a powerful tool to "internationalize" the 
counterfeiting of brands and products. Last but not least, "made in Italy" and "Italian Sounding" 
must be protected from foreign usurpation. EXPO 2015, that in the next six months is expected to 
bring 20 million visitors to our Country, is an exceptional vehicle for the promotion of Made in Italy 
and of our nation as a whole in an international environment. Made in Italy is our pride but even 
more, it is the driver of Italian development. The title of Expo 2015, "feeding the planet, energy for 
life", introduces many themes, all of them being fundamentally based on ethics: in fact, it's not 
ethical to infringe agro-alimentary rights, especially the famous "Italian Sounding" in which 
unauthorized product are sold under an Italian name. Lack of respect for the Country of origin 
costs Italy 60 billion euro per year in lost turnover, while the damage to the Italian Agro- 


24 
Vito Cappellini (edited by), Electronic Imaging 6 the Visual Arts. Eva 2015 Florence ISBN 978-88-6655-758-6 (print) 
ISBN 978-88-6655-759-3 (online) O 2015 Firenze University Press 


alimentary sector accounts for 6 million euro per hour. EXPO 2015 gives Milano a unique 
opportunity to re-launch Italy as a protagonist in the promotion of fair and respectful competition 
regulation, which must recognize and protect diverse geographical-cultural origins worldwide. 


To sum up: different instruments for each sector are needed, but always under the wise lead of a 
unique large institutional orchestra. 
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Interpreting heritage: Where technology meets the audience 


Dr. Margaretha Mazura, Secretary General EMF — the Forum of e-Excellence 


Dr. Kaja Antlej, eCultAmbassador, currently Postdoctoral Researcher, Centre for Creative and Cultural 
Research (CCCR), Faculty of Arts & Design, University of Canberra 


In heritage interpretation digital technologies have been in use for some decades as they enable 
unique opportunities for enhancing visitor/user experience. Many interpreters, who are using digital 
technologies as possible tools for transmitting heritage meanings to a wider public, have noticed that 
their implementation is a complex process. It requires not only knowledge from both heritage and 
technology poles but also good communication skills in order to establish a successful collaboration 
between content (heritage institutions) and technology providers. Several times communication 
challenges between engineers and cultural professionals have been summarized in John Gray's 
metaphor paraphrased as: »Technology comes from Mars, culture from Venus«. 


To tap on the opportunities digital technologies offer, they must be integrated into an overall 
strategy of museums and other cultural heritage institutions. But how to select the right technology 
for the interpretation of your collection? And why? 


Cultural heritage artefacts are still today mostly seen in their isolation in one space like a museum. 
Visitors want more tailor-made exhibitions or experiences that correspond to their interests. What is 
lacking, and what hampers to engage young people in museums, is their contextualization and the 
lack of user centricity. Two EU research projects that explored (and explore) the use of new 
technologies to engage users are: CHESS - Cultural Heritage Experiences through Socio-personal 
interactions and Storytelling (www.chessexperience.eu) and TAGCLOUD (www.tagcloudproject.eu). 
CHESS was successfully deployed at the Acropolis Museum in Athens: specially adapted tablets allow 
the visitor to see the artefacts in their original environment, with the (original) colour paint on them. 
Visitors can select a circuit to their interest. Technology used are Augmented Reality paired with 
storytelling via the tablet. TAGCLOUD uses mainstream smartphones to convey storytelling at 
cultural heritage sites like the Alhambra in Granada. Users can share their experience and knowledge 
via social media that are linked to the App thus enhancing the storytelling by their own stories. 


Another way to include users/visitors or other knowledgeable people (e.g. collectors) to interpret 
heritage is crowdsourcing. The scientific article “Crowdsourcing in the Cultural Heritage Domain: 
Opportunities and Challenges"! points to the opportunities but also caveats when using 
crowdsourcing. 


More documentation and practical guidance on technologies that facilitate interpreting cultural 
heritage can be found at: http://www.ecultobservatory.eu/basic-page/ambassadors-toolkit 


BACKGROUND: 


The eCult Observatory studies museum needs and available technology solutions and tries to give 
guidance to both sides. eCult Ambassadors, a network of trained persons with special knowledge in 
cultural heritage and technology, can act as intermediaries and help selecting a technology. For more 
information, please contact us: www.ecultobservatory.eu/contact. 


1 Johan Oomen, Lora Arroyo, http://www.iisi.de/fileadmin/IISI/upload/2011/p138 oomen.pdf 
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MUSEO NAZIONALE DEL BARGELLO: 
*MNEMOSYNE - SMART MUSEUMS" PROJECT 


Ilaria Ciseri 
Museo Nazionale del Bargello 


Florence, Italy 


<ciseriQpolomuseale.firenze.it> 


Last February at the Museo Nazionale del Bargello, in Florence, has been presented a three-year 
project funded by the European Commission and Regione Toscana and realized by the MICC 
(Centro per la Comunicazione e Integrazione dei Media) of the University of Florence, under the 
direction of Alberto Del Bimbo and Thales Italia SpA. The project is very interesting and 
innovative: for the first time, in Italy, a network of fixed and mobile sensors analyse visitor's 
behaviour inside the museum and index each visitor's interest to discover related artworks. The 
right of privacy is guaranteed. The system consist in a multimedia touchscreeen collocated in the 
Donatello Room, the main room of the museum, where are on display masterworks by the most 
important Renaissance sculptors as Donatello, Verrocchio, Desiderio da Settignano and Luca della 
Robbia. The Mnemosyne touchscreeen shows a selection of 10 sculptures and and offer a 
multimedia content. 


This sperimental project is the first interactive kind of multimedia product conceived to provide 
each user a personalized museum experience. 


To increase digital technologies is one of the main aims of the Bargello Museums to improve the 
didactic apparatus and to permit a modern approach by the visitors. 
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HIGH QUALITY ARCHIVE PROJECT 
FOR POLO MUSEALE FIORENTINO: 
EXPLOITATION ACTIVITIES 


Vito Cappellini, President MICC - University of Florence, Italy 


Takayuki Morioka, Director DIS Project, Hitachi Ltd., Yokohama, Japan 


Marco Cappellini, CEO Centrica S.r.l., Florence, Italy 


SUMMARY 


The Project on *High Quality Digital Archive for Polo Museale Fiorentino", developed by MICC - 


DINFO - University of Florence, Hitachi Ltd. and Centrica S.r.l., in cooperation with Museums of Polo 


Museale Fiorentino, is continuing its activities along the planned lines. Many important art-works have 


been digitized at very high quality: 
1. Leonardo da Vinci, Annunciazione, Uffizi 
. Leonardo da Vinci, Adorazione dei Magi, Uffizi 
. Leonardo da Vinci, Battesimo di Cristo, Uffizi 
. Michelangelo, Tondo Doni, Uffizi 
. Tiziano, Venere d'Urbino, Uffizi 


2 

3 

4 

5 

6. Caravaggio, Bacco, Uffizi 
7. Caravaggio, Medusa, Uffizi 

8. Piero della Francesca, Dittico di Urbino, Uffizi 
9. Bronzino, Ritratto di Eleonora di Toledo, Uffizi 
10. Botticelli, Madonna del Magnificat, Uffizi 

11. Botticelli, Primavera, Uffizi 

12. Botticelli, Nascita di Venere, Uffizi 

13. Giotto, Madonna di Ognissanti, Uffizi 

14. Raffaello, Madonna del Cardellino, Uffizi 


15. Raffaello, Madonna della seggiola, Palatina 


16. 
17. 
18. 
19. 
20. 
21. 
22: 
23. 
24. 
25. 


Lega, Il Canto dello Stornello, Museo d'Arte Moderna 
Fattori, Libecciata, Museo d'Arte Moderna 

Correggio, Adorazione del Bambino, Uffizi 

Andrea Del Sarto, Madonna delle Arpie, Uffizi 

Paolo Uccello, Battaglia di San Romano, Uffizi 
Giovanni Bellini, Allegoria Sacra, Uffizi 

Filippo Lippi, Madonna col Bambino e Angeli, Uffizi 
Tiziano, Flora, Uffizi 

Parmigianino, Madonna dal collo lungo 


Simone Martini and Lippo Memmi, Annunciazione tra i santi 


Ansano e Margherita, Uffizi 


26. 
21: 
28. 
29. 


Andrea Mantegna, Trittico, Uffizi 
Lorenzetti, Presentazione al Tempio 
Gentile da Fabriano, Adorazione dei Magi, Uffizi 


Pontormo, Cena in Emmaus, Uffizi 


Several technological improvements have been added since the starting of the Project. First of all the high 


quality was further important by means of the three advanced techniques: 


- very high resolution; 
- color correction (calibration); 


- geometric distorsion control. 


Some activities developed in last year are described in the following. 


Two important activities developed by Hitachi Ltd. regarded Uffizi Virtual Museum (UVM) Exhibition 


in Guam, USA and Permanent Exhibition in Hitachi’s showroom in Yokohama, Japan. 
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The UVM Exhibition was held by the Honorary Consulate of Italy in Guam (at the Under Water World 
Complex in Tourism) for 49 days from 10 August to 27 September in 2014. It was the first for Guam to 
hold the Exhibition and it was very well received by travelers as well as local people. The total number 
of visitors recorded almost 6,000. 

For what regards Permanent Exhibition, DIS Digital Museum continues (since its opening in October 
2012) to show UVM contents and other replica & digital data newly digitized. 

The average number of visitors is 50 per month and word-of-mouth recommendations have been 
increasing. 

Presentations by MICC and Hitachi Ltd. have been done in October 2014 at KNOWDLEDGE 
CAPITAL in Osaka in the framework of *Art and Science New Renaissance" Event. 

Further Uffizi Touch Studio by Centrica S.r.l. has been part of Engineering Ingegneria Informatica 
booth during European Utility Week (Amsterdam, 4-6 Nov. 2014). 

Uffizi Touch© Cloud by Centrica S.r.l. has been demonstrated in Saint Petersburg (ITMO University 
27-28 October 2014) and at CEBIT in Hannover (20 March 2015). 
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NEW TECHNOLOGIES 
& APPLICATIONS 


INTERNATIONAL CLUSTER APPROACH AS A FAST 
TRACK TO PRACTICAL REALIZATION OF DISRUPTIVE 
INNOVATIONS IN CULTURE 


Konstantine Karczmarski 
University Innovation Office 
ITMO University 
St.Petersburg, Russia 
konstantine.karczmarczyk@gmail.com 


ABSTRACT 
Several industries, that to a certain extent define how our life looks, have been striving to find a new 
and disruptive model that fits into reality of the information age. Those are everybody located 
between us and creators of any object of culture: museums and media companies, telecoms and 
theaters, film production companies and even travel agencies and, of course, quite a number of 
great tech startups trying to serve culture. Meanwhile, one of the greatest concepts - “culture as 
socio-economic driver" has been underperforming. One of the main reasons is that disruptive 
innovations require building entire business and social processes that connect new technologies in 
consistent new value chains, while public, academic and business efforts have been concentrated on 
isolated innovations. 
Content 

1. Introduction 

2. Sustainability, efficiency, disruptive innovations. Which one is the most important for EU? 

3. Innovations in culture. Why their value is not monetized? 

4. Multi-disciplinary and multi-industry integration needs 

5. Building new value chains in culture 

6. Involvement of academia. Major role, limits and relationships with businesses 


7. Internationalization and “distributed cluster” model 


8. First practical results and conclusions 


32 
Vito Cappellini (edited by), Electronic Imaging & the Visual Arts. Eva 2015 Florence ISBN 978-88-6655-758-6 (print) 
ISBN 978-88-6655-759-3 (online) © 2015 Firenze University Press 


"The Reinvention of Print in our Digital World: It's Personal, 
Powerful, Permanent" 


Andrea de Polo Saibanti 
Dept. Digital Imaging and R&D 
Fratelli Alinari IDEA SpA 
Firenze, Italy 
andrea@alinari.it, http://www.alinari.com 


Abstract — Traditional film photography has declined considerately in the last decade. 
However print is not dead. Although traditional print on photographic paper has also faced 
recently a major decline, digital prints have seen recently, a small but significant increase in 
usage. Print materials can't create the flexibility of a digital platform, but digital content doesn't 
have the same tactile human response as paper, nor can it be used for specific branding needs 
such as product packaging. In this presentation we will discuss about the best options and 
opportunities for photographers, artists, designers, graphics, students, museum people about 
various solutions available today in the printing world going from the personal to the 
commercial evolution and sustainability. Although social media and technology will always 
grow, it does not mean that printing is dead. 


INTRODUCTION 


Today, taking digital pictures is as simple as 1+2+3. Moreover, technology has become 
extremely inexpensive and the usability of most digital devices has become truly attractive 
even for young users. As consequence, those days, billion and billion of pictures are taking 
daily. However only a small fraction of them is printed or even properly stored and preserved 
for long term conservation. In terms of statistics, every day a human being see between 3000 
to 15000 advertisement images, NY Times research (on TV, in the street, on the mobile phone 
and so on). In 2014, perhaps 90m traditional cameras will be sold - and close to 2bn phones 
and tablets with cameras. There will be over 2bn iPhone and Android smartphones on earth 
by the end of this year: with perhaps 4bn people on earth with mobile phones, there are at 
least 3bn camera phones and probably over 3.5bn. Over 1.5bn new photos are shared every 
day on Facebook, WhatsApp and Snapchat alone, which equates to about 550bn a year, and 
this is growing fast. The number of photos we take each year has sky rocketed. In fact, in a 
recent presentation by Yahoo!, it was claimed that as many as 880 BILLION photos were 
taken in 2014. So what is the main reason why with so much digital content around, people 
print lately less and less than before? 


The boom of digital photography 


Today, the smartphone in your pocket has a high-quality digital camera. Everyone -- not 
just artists -- is a photographer, and the explosion of photos taken annually proves it. For 
example the number of picture taken every year in 1999 was 86B and 380B in 2011! In 
addition, 8396 of digital camera photographers edit their pics with a computer. And today we 
capture a new photo every 2 minutes, as humanity did as a whole in the entire 1800s! 
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How Many Photos Are Taken Each Year? 


A All Photos A Analog Photos 
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E 2 


# of Photos Uploaded & Shared per Day 
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8 8 8 
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Pricing for computers, mobile phones, digital cameras, video devices have dropped in the 
last 10 years by over 70%. High quality camera phones can cost around 800 euro (but gold 
versions could go as high as 10.000 euro!). However inexpensive cameras with a decent 5 
megapixel camera can still go as low as 70 euro. And with a 5 megapixel camera, you can still 
get very good pictures ready to be printed in good quality on a A4 photographic type of paper. 
However printing is one of the main problems that the digital industry if facing today. Why? 
Today people prefer to store their images on the Cloud, to post them online, on social 
networks (Instagram, Facebook, Flickr, etc) or just save them on a local disc, rather than print 
any of them. In the past, let's say until mid *90's, especially with traditional photography, it 
was quite normal to print photos, put them in an album, enlarge them, put them on the wall. 
Today, although we capture many more images than before, printing them is a totally 
different story. 
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From Analog to Digital 


When Robert Burley (an associate professor at Ryerson University in Toronto) began 
documenting the global implosion of the silver-halide roll-film industry in 2005, he used an 
analogue camera. A digital one would have been a quirky choice for his style, unable to 
deliver the same precise results he was used to after decades of photographing architecture 
and landscapes. But as Mr Burley's journey progressed, he watched the ecosystem of film 
rapidly dissolve around him. "I was starting to feel like a blacksmith," he says, recalling the 
large-format camera kit he would unpack in order to capture his waning industrial subjects. 

Kodak's bankruptcy filing in January 2012 was a result of decades of mismanagement. But 
it was also the victim of rapid technological change for an industry based on chemistry and 
large-scale production of an obsolescent good. The spike in silver prices was no help, either, 
for a product that must needs use it. 

Consumers and professionals ditched film first. Then health-care services, which used it for 
X-rays, shifted to digital scans. The final blow came with the film industry's switch to digital 
projection. IHS iSuppli, a supply-chain analysis firm, estimates filmmakers consumed 4m 
kilometers of film each year for the distribution of prints at its height. That was just a few 
years ago. By 2012 this plunged by two-thirds. In 2015 it will be next to nothing. Mr Burley 
says that after years of talking with the workers, chemists and engineers that ran the plants he 
foresees a tipping point beyond which consistent quality photographic film will be impossible 
to make because of the scale necessary to maintain operations. 

That point has not yet been reached. Polaroid factories in Massachusetts may be 
abandoned, but those in Enschede, the company's former European headquarters in the 
Netherlands, live on. That is thanks to the Impossible Project, which aimed to reboot instant- 
film production using original equipment (as well as a fair amount of reverse-engineering, or 
reinventing, lost secrets). With the expertise and hard work of a handful of people it 
succeeded, and has shipped millions of units of print film, including new variants that go 
beyond anything Polaroid made. It relies partly on Ilford, a British manufacturer of black-and- 
white film based in Mobberley (and also documented in the book). Ilford has so far survived 
bankruptcy and upheaval. But the Impossible Project as a whole depends on ancient 
equipment, a limited term lease and chemicals and processes provided by other firms (1). 

With digital technology we can for example extend the dynamic range beyond the visual 
spectrum and manipulate in Photoshop or with the automatic histograms, in our smartphones 
or digital cameras, the scenes, to settings and tonal range not available in traditional analogic 
form. So imaging and photography is become today much affordable, easy to use and easy to 
operate. As consequence, in the last decade, digital imaging has clearly surpass traditional 
photography in many sectors: from photography to desktop publishing, from movies to 3D 
animation, from traditional silver halide based prints to ink-jet, laser jet, thermo prints. 

Chart below shows how digital cameras killed analog cameras and how the selfie culture 
(almost) killed digital cameras after that! 
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1) Analoge photography has been killed rapidly by the digital camera generation. 
2) The Digital camera business got "almost" killed by the smartphone selfie generation :) 


According to CIPA's camera production statistics, the recession of the camera market has 
slowed down a little. It decreased just 32.9 % until October 2014 compared to the same period 
last year. 

Questionably, nevertheless, is that the decline of the margin product SLR increased it's 
momentum. 25% this segment shrinked in 2014 so far - without hope that this decrease could 
be recovered by the sales of mirror free compact systems. 

What impact this development has on the total photo market, becomes even more clear if 
we look to the sales ratios of the German photo market: In 2013 the share of cameras and 
lenses counted for not less than 85% according to European market research company GfK 
(sales of photo finishing products excluded) — a relation that most probably is not 
substantially different in other countries. (2) 


Let's print then? 


Save our memories and printing are two important actions that are currently quite missing 
in the digital eco-system workflow. Traditionally photography is not dead. In fact, Kodak's 
U.S. professional film revenue grew recently by 20 percent, and, according to Scott DiSabato, 
As Kodak has improved its Portra color negative films in recent years, the company has found 
ways to use components that are also used in consumer film and motion picture film 
manufacturing, improving efficiency and sharing costs. 

U.S. marketing manager for professional film, pro film sales in Europe and Japan are also 
showing signs of improvement. - See more at: Kodak's competitors have also recently 
suggested that film remains a profitable business. One manufacturer, Lomography, is seeing 
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substantial growth. Michael Bain, a U.S. representative for Harman Technology, the company 
that manufactures Ilford films, reports that Ilford's sales were up. Ilford saw the most 
substantial growth in its 120 roll film and sheet films, including its ultra large format, or 
"ULF" films, for which it takes custom orders once a year. 

Clients are also asking photographers to shoot film for certain jobs. Pari Dukovic, who 
shoots film exclusively, has been working for magazines like New York and The New 
Yorker, and says that editors come to him for the grainy, painterly look he achieves using 
high ISO black-and-white and color films. J.Crew asked Jake Chessum to shoot film for a 
project for one of their specialty stores in New York City a couple of years ago, and a few 
magazine editors have asked that he shoot film recently. 

Photo students continue to use film, and educators continue to teach darkroom printing. 
Stephen Shore, who is head of the photography department at Bard College, says that a few 
years ago incoming students began showing up with no film or darkroom experience. More 
recently, during an introduction to the program he gives at the beginning of each year, when 
he told the students their education would be analogue until they were juniors, “spontaneously 
a bunch of students started clapping." (3) 

Today the industry is try to educate people, once again, about the importance to save their 
content on proper format (and savvy place) for long term storage (such as TIFF or jpeg2000 
in a DVD and tape), use cool temperature, low light and low humidity in the data storage. 

In addition, printing is become again important as ink, paper and printing speed have all 
become economically driven and more affordable than in the past. Thanks to the latest 
printers in the market it is today quite easy to operate those machine mostly unattended, and 
this is great to run printing operation also on new locations like museums and galleries where 
the concept of print on demand is often seen as a great option to generate new revenues. 
Among the various companies and technologies available in the market, today it is quite 
popular the success of the Epson Stylus Pro 9900 printer. The Epson Stylus Pro 9900 Printer 
represents the highest level of innovation and performance for fine art and commercial print 
applications. Designed for 44" wide printing, the Pro 9900 incorporates a state-of-the-art 
MicroPiezo TFP print head with an ink droplet size of 3.5 picoliters. Leveraged with Epson's 
new AccuPhoto HDR screening technology, you can count on your most critical work to be 
rich, detailed, and accurate. 
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Epson's latest High Dynamic Range pigment inks offer exceptional quality for true 
archival printing. When used with Epson Professional Papers, color images are rated up to 
200 years. Black & white printing lasts even longer! Eleven individual ink cartridges offer a 
super wide color gamut and eliminate waste. 

In addition, The Epson Stylus Pro 9900 utilizes Optimal Black Ink Technology to 
automatically switch between Photo and Matte Black Inks. This delivers media-specific 
density for exceptional contrast, shadow detail, and line form. The black ink selected is easily 
confirmed on the front control panel. 

Capable of handling virtually any media type up to 44" wide, the Epson Pro 9900 Printer is a 
wonderful choice for large format photographers and image professionals world-wide. 
Capable of printing images up to 44" wide, the Epson Pro 9900 is perfect for making fine art 
gallery prints and banner productions. In living color or dramatic black & white, bring your 
wide format vision to life. 

Epson's latest High Dynamic Range (HDR) pigment inks offer exceptional quality for both 
archival printing and short-run proofing applications. Color prints on Epson Professional 
Media are rated stable for up to 200 years (black & white prints last even longer). The short- 
term stability of the inks is equally impressive. From proof to print, these inks were developed 
to meet the demands of the professional image maker of today and tomorrow. 


ol Epson UltraChrome* HDR Ink Technology 
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All 11 individual ink cartridges are available in 150, 350, and 700ml capacities to help control 
costs and eliminate waste. Use the inks you need most and reduce your impact on the 
environment. Cartridge sizes can be mixed and matched on the fly. Controlling ink use by the 
job or by the client has never been easier. 

By combining Black, Light Black, and Light Light Black Inks, the Epson Pro 9900 produces 
film-quality black & white photos from color or grayscale files. From the darkest black point 
to the brightest white point, prints offer smoother tonal gradations without sacrificing 
contrast. With the Epson Pro 9900, wide-format, fine art black & white imaging rivals the 
very best fiber prints! 
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Conclusions: 


Traditional photography is not dead. Long life to analog photography! However digital 
photography, digital imaging and digital prints is the standard solution for imaging today and 
probably for the future. However save our data in a reliable and harvested and compatible 
solution is truly a must for everyone. Plus print our memories is indeed important. Memory 
and data can be volatile. Prints are not. Prints, stored well can live for many decades. An their 
quality are today as good (of not better) than traditional analog prints. The prints are the best 
way to share our emotions, feelings, momentum with our relatives, friends and colleagues. 
Facebook, Flickr and so on are online social media that actually do not guarantee the storage 
of your content forever. Plus, to enjoy them, you must connect every time online. And 
sometimes your best pictures might be gone or your account my be lost or suspended. So 
printing is indeed a very important action to be made in our digital workflow and eco-system. 
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Abstract — The paper describe a possible approach for passive authentication of printed 
objects. It is relates to a method for antiforgery marking and authentication of printed products 
called ViSeQR®. 


SUMMARY 


The term "forgery" is generally used in a wide sense and comprises any production of a 
product that imitates the appearance of an original product so as to induce a consumer into 
error. 

Described in the present work is an antiforgery approach for traceability of printed 
products. More in particular, the proposal technique regards a system for rendering a printed 
product non-riproducible, thus preventing unauthorized reproduction thereof: in this way, it is 
possible to protect the printed product itself. 

In the current state of the art, the phenomenon of forgery of products is a problem that 
continues to increase and hits transversely all sectors of production, and for its very nature 
shifts the equilibrium of the markets and affects competition with extremely high costs due 
both to the loss of earnings and to the loss of image of the manufacturer of the product itself, 
caused by the poorer quality of the goods present on the market. 

To tackle this problem companies invest considerable amounts of capital in the protection 
of their own rights and in technologies for counterfeiting forgery. Many technologies are 
available, amongst which there is a widespread use of optical and electronic technologies, 
RFID technologies, techniques of electronic marking or "information hiding", holographic 
techniques and 2D barcodes, techniques of chemical or mechanical marking, and the like. 

The basic idea behind the proposed approach, is that while the information carried by the 
QR code is easily reproducible e.g. using a photocopier, the frame printed around the QR 
code carries a sort of fingerprint of the printer that has physically generated the image, and 
such fingerprint can't be reproduced. The method used for the extraction of the fingerprint of 
the printer is based on extremely effective passive authentication techniques, based on image 
processing, that are the core of the know how of ViDitrust. Being these techniques “passive”, 
they arrive to uniquely identify the printer with no need for special paper or inks, and with no 
additional cost for the printing process. 
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Abstract — In recent years the significant technological development is changing many 
aspects of the daily life, in a number of basic aspects such as health-care. In the past 
each person received appropriate treatments by specialist medical staff and 
examinations from laboratory tests. On the contrary nowadays the patient is assuming 
an active role and is much closer to both examination and therapy phases. In this 
communication we will review this new approach, with regards to new technologies like 
Point-of-Care Diagnostics for low-cost and user friendly health monitoring and 
Nanomedicine for early and minimally invasive treatment of diseases like cancer. 


In recent years the significant technological development is changing many aspects of the 
daily life of each person, not only for the adoption of new technologies, but also for a 
different approach in a number of basic aspects of life such as health-care. In the past each 
person received appropriate treatments by specialist staff, such as doctors, nurses, which took 
care of therapies rather than medical examination, from laboratory tests to radiological 
examinations. The presence of a "mediator" kept the patient away from the specific 
technologies which were used. On the contrary nowadays the patient is assuming an active 
role in both examination and therapy phases. An example is represented by the measure of 
blood glucose level for insulin treatment. This activity requires a direct involvement of the 
patient. Therefore the patient turns in a technological user, as a caregiver, even if with a 
different background, different knowledge, different context of use, different needs which 
require a different interaction with the application, the device, the process. These new 
interactions must be studied and introduced at level of design of each new device, that 
together with medical aspects takes also in account new concept such as accessibility, 
usability and acceptability. 

In hospital point-of care diagnostics, the measurement of biomarkers specific for definite 
pathologies is performed close to the patient bed and the information should be immediately 
transferred to the physicians within a local network so as to allow the formulation of the right 
diagnosis or the identification of the correct therapy in a very short time; this is particularly 
necessary in intensive care unit where the conditions of the patients must be monitored 
continuously or very frequently on long periods. For example, measurement of 
immunosuppressants in transplanted patients in the first two weeks after transplantation or 
measurement of specific biomarkers for the discrimination of viral and bacterial sepsis and 
for the fast identification of the origin of infections are just two of the emerging needs. 
Bringing diagnostics and therapy closer to the patient, also means finding more precise and 
less invasive approaches to fight widespread diseases like cancer, which remains one of the 
leading causes of death. In this regard, the advent of Nanotechnology has revived the hope 
for selective alternatives to invasive medicine. In particular, gold nanoparticles (GNPs) have 
received considerable attention. In addition to their good biocompatibility, ease of 
preparation and stability, their optical features when excited e.g. by laser light are ideal for 
applications in Nanomedicine. In this regard, we are exploring complementary pathways to 
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deliver these particles to tumors by the interplay of passive and active strategies. For instance, 
GNPs can be functionalized to target and bind with specific biomarkers of cancer cells in 
order to act as highly selective contrast agents for the early detection of tumors. Moreover, 
under laser illumination, they can provide tumor ablation by selective photothermal treatment 
or by means of intracellular release of chemotherapic agents, which offer less invasive 
alternatives to currently available treatments, such as surgery, radiation therapy and 
chemotherapy. 
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Abstract 


Within the recent 10 - 15 years a large number of challenging 3D scanning projects has been 
realized, demonstrating that close range 3D surface scanners, based on optical triangulation, can 
be used not only in special labs with controlled conditions, but may be operated in remote areas 
and under difficult environmental conditions. The paper will present the most important 
features of selected scanning technologies, paying special attention on the latest progress on 
photogrammetric technologies and on different approaches for mass digitization. It also will ask 
for reasons why 3D scanning is still far away from being the standard technology for 
documentation, archiving and scientific research of artefacts in CH. 


INTRODUCTION 


In the 80" of the last century the first prototypes of 3D surface scanning systems, based on 
optical triangulation, have been presented. For about two decades the development of most of 
these systems was mainly driven by the needs of industrial metrology, where optical 3D 
measurement systems have replaced classical tactile systems for a lot of applications. 


Nowadays, various close range 3D scanning technologies are also available for a wide 
range of applications in cultural heritage. The most important techniques are based on 
photogrammetry, stereometry, laser scanning, structured light approaches and phase shift 
technologies. They offer resolutions in the sub-mm to um-range for small and medium sized 
objects, with measuring ranges from um to several meters. Nevertheless, the different 
techniques vary strongly in important performance features as resolution, accuracy and data 
quality, in system parameters as weight and size, flexibility and transportability, and finally in 
the complexity and costs of the equipment and required skills for its operation. 


CLOSE RANGE 3D SCANNING TECHNIQUES 


Most people are familiar with the concept of optical triangulation, because the human 
vision system with its two eyes is based on a stereoscopic set-up. A straight forward technical 
realization of our vision system may consist of 2 digital cameras (see fig. 1) and an image 
processing system to correlate or match the stereoscopic images. All geometrical and optical 
parameters of such a stereometric set-up can be determined in advance by a calibration 
procedure. Thus, each pair of stereoscopic images, recorded with this set-up, allows the 
calculation of 3D information by correlating homologous feature points in the stereoscopic 
images. It shall be pointed out, that accuracy, resolution and reliability of the evaluated 3D 
data strongly depend on both, the visibility of feature points and the quality of the used 
matching and correlation algorithms. 


44 
Vito Cappellini (edited by), Electronic Imaging 6 the Visual Arts. Eva 2015 Florence ISBN 978-88-6655-758-6 (print) 
ISBN 978-88-6655-759-3 (online) O 2015 Firenze University Press 


In photogrammetry a single digital camera is used to record a sequence of images. The 
optical parameters of the camera (interior parameters) and the position and orientation of the 
single images (exterior parameters) are determined by bundle adjustment algorithms, which 
are also based on matching and correlation procedures. Thus, photogrammetry is a self- 
calibrating technique. It shall be pointed out, that this is valid only in case of using 
appropriate scaling bars for each measuring sequence. 


To guarantee the required accuracy and reliability for applications in industrial metrology 
the correlation of homologous points must be calculated with sub-pixel accuracy, typically in 
the range of 1/10 of a pixel or even better, which is difficult to achieve on the texture of the 
measurement objects, only. Therefore, in the past the use of photogrammetric and 
stereometric systems in metrology was restricted on the determination of 3D coordinates of 
index marks applied on the surface of the measuring object. 


To overcome the many problems and restrictions of correlating images just by image 
processing techniques, topometric 3D surface scanners have been developed, which are based 
on a structured light approach, where a laser/pattern/fringe projector is integrated into the 
sensor set-up. The simplest configuration is to replace one camera of a stereometric set-up by 
the projection unit (see fig. 1). This allows to create an unambiguous indexing of all object 
points and a reliable and quantitative calculation of 3D data. Topometric scanners can be 
calibrated in advance; thus the scanning process itself does not require the use of scaling bars. 


Figure 1: Technical realisation of a stereoscopic set-up with two digital cameras (left); 
principal set-up of a fringe projection system (right) 


Most of the topometrical systems, based on structured light techniques, are either using a 
random pattern or a fringe projection technique. The random patterns are analyzed by spatial 
correlation techniques, which results in a strong low-pass filtering (smoothing) of the 
calculated 3D data. On the other hand, these techniques are instantaneous, because the data 
acquisition can be based on the recording of only a single image. 


High definition fringe projection techniques are mainly based on the phase shift technology 
Instead of recording only one image, a sequence of fringe patterns is projected and recorded 
for each scan. The main advantages of these techniques are: 


- It allows the separation of fringes and background structure by basic arithmetic operations 


- As the local phase ọ of the fringes can be calculated with highest accuracy without using 
spatial correlation algorithms, 3D coordinates can be determined with sub-pixel accuracy 


- The reliability of the resulting 3D data can easily be determined by the fringe contrast 
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HIGH DENSITY PHOTOGRAMMETRY vs. FRINGE PROJECTION 


Due to the recent significant progress in image indexing and matching algorithms new 
approaches for high density photogrammetry have been developed, e.g. Structure from 
Motion (SfM). An overview and a comparison of different dense image matching algorithms 
is given in [1]. Although some of these tools still require images, which are already 
undistorted and pre-orientated, the automated generation of high density 3D data based on a 
set of photogrammetric images is on its way. One major advantage of high density 
photogrammetry results from the fact, that the data recording does not require special 
equipment, which might be bulky and expensive, but can be based on off-the-shelf cameras. 


Therefore, people are already claiming that the future of close range 3D scanning will be 
based just on smartphones and tablet computers without requiring any further equipment and 
special skills from the operator. No doubt, further improvements of images sensors and 
related software tools, which can be expected within the next few years, will allow an 
automatic generation of high density 3D data based on images recorded by those devices. 


But what's about accuracy, resolution and reliability of the calculated data? 


Assuming that all problems with image indexing and matching, automated orientation and 
correction of images are mainly solved, there will be still some facts, which restrict the 
performance of high density photogrammetry, in particular when consumer goods are used. 


- Smartphones and tablets are not designed with respect to the requirements of metrology, in 
particular regarding mechanical and electronic stability. 


- Changes in the mechanical and electronic parameters of the recording devices during the 
data acquisition will cause inconsistencies in the recorded images. 


(For this reason, high definition cameras are used in industrial photogrammetry) 


- The accurate calibration of the calculated 3D data requires certified scaling bars and 
related experiences of the operator. 


- There are a lot of objects without or with only limited texture information, which does not 
allow reliable image matching for the whole object. 


- Any kind of correlation technique causes an intrinsic low-pass filtering, which strongly 
limits the structural resolution of the calculated 3D data. 


The following figures 2 — 3 demonstrate the comparison of two sets of 3D data of a 
fragment of an ancient vessel (University of Graz, Institute of Archaeology, inv. G741 [2]); 
one has been recorded with SfM, the other one with fringe projection technique. The 
parameters of the recording equipment and the scanning process are listed in table 1. 


Fringe Projection Structure from Motion 
Scanned by Breuckmann GmbH ArchaeoAir 
Recording equipment smartSCAN HE color Olympus Pen MI 
Camera resolution 2x5mpx 12 mpx 
Resolution in image domain 40 um variable 
Number of scans / images 16 200 
Calibration with certified calibration plate | none 
Accuracy of calibration 15 um - 
Total acquisition time 15 minutes 10 minutes 
Data processing time 10 minutes 60 minutes 


Table 1: Parameters of recording equipment and scanning process 
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Figure 3: 3D data of fragment G741, left: fringe projection, right: SfM 


The main results of the quantitative comparison are: 


- The resulting texture data are quite similar, with that recorded by fringe projected being 
slightly sharper. 


- There is a strong difference in the structural resolution of the 3D data, with clear advantage 
of fringe projection. 


- The dimensions of the calculated 3D models differ by about +/- 0.5 mm, which is not 
surprising, as no calibration has been carried for SfM. 


ESTABLISHING 3D SCANNING TECHNOLOGIES IN CH 


Within the recent 10 — 15 years a large number of challenging 3D scanning projects have 
been realized, demonstrating that close range 3D surface scanners, based on optical 
triangulation, can be used not only in special labs with controlled conditions, but may be 
operated in remote areas and under difficult environmental conditions. Nevertheless, 3D 
scanning is still far away from being the standard technology for documentation, archiving 
and scientific research of artefacts in CH. Will this situation be changed by the latest progress 
on photogrammetric technologies and the development of special equipment for mass 
digitization? 
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We believe so. But not because we are convinced, that high density photogrammetry and 
mass digitization systems will be the only future of 3D scanning in cultural heritage from a 
technical point of view. However, the availability of reliable and easy-to-use software for 
generating high density 3D data based on photogrammetric images will help users, to become 
more familiar with 3D technologies without spending lots of money for high definition 
equipment. They will learn for which kind of applications these techniques are suitable; by 
making their own experiences with 3D scanning users will get a better feeling for the special 
features and restrictions of the different approaches. This will close the lack of experience 
with 3D scanning systems, which in our opinion is one of the main reasons for the missing 
acceptance of these technologies in CH. 


We also believe that innovative algorithms for image indexing and matching will improve 
topometric high definition 3D scanning systems, which are the essential ones for 
documentation, archiving and all kind of measuring purposes. By improving the flexibility 
and speed of data recording and evaluation, the acceptance of these technologies will be 
further increased. 


Systems for the semi-automatic mass digitization of small objects like coins and fragments 
are available since about 10 years; these systems are transportable and available for 
reasonable prices. Systems for medium sized objects up to about 50 cm are under 
development. Up to now, the size and the costs of these systems are quite high; the 
installation requires special skills and takes time. Thus, the collections must be brought to the 
scanning station in most cases, which may cause additional problems, e.g. for getting the 
necessary permissions and organizing the transportation. For large and complex objects, 
robot-assisted scanning stations, as they are state of the art for industrial applications, are 
suggested. However, lots of security problems must still be solved, before autonomic robots 
may be operate in front of most valuable CH objects. 


Moreover, one has to recognize that the digitization of the single objects is only one part in 
the complete complex workflow of mass digitization. Organizing the access to the objects, 
certifying correct handling and transportation as well as the necessary documentation of all 
steps sometimes takes more time than the scanning process itself. From this point of view, 
one must carefully balance the advantageous of mass digitization systems against their costs, 
restrictions and limitations in every single case. 
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Abstract - This paper discusses the collaborative interactions of media facades. Through the use 
of the media facade at he University of Applied Sciences Berlin, different technologies and 
methods were analysed, to solve specific problems while working collaboratively in a 
team. The goal was to add and optimise collaborative interactions in existing single or 
multi user applications. 


1 INTRODUCTION 


The urban landscape of today's cities is often characterised by media facades. Originally they 
were used for public viewing and advertising. Today they have expanded to include multiple 
functions as interactive communication displays. An important aspect of interacting with a 
media facade is to acknowledge the possibilities as well as the restrictions the public has 
towards them and thus allow the development of a more active and collaborative interaction. 


The different techniques used in the media facade at the HTW Berlin offer the possibility to 
interact in a single or multi-user mode, as well as first collaborative interactions. Since the 
first classic games such as Pong, Tetris or Frogger, the visualisation of data from information 
systems has been added to the media facade. In the future this should enable or facilitate 
students as well as staff of the HTW Berlin to collaboratively work or interact with the media 
facade. 


2 FKIMEDIA FACADE - TECHNICAL BACKGROUND 


The media facade is located on the research and training centre building for Culture and 
Computer Science (FKT) on the Wilhelminenhof campus of the University of Applied 
Sciences Berlin (HTW Berlin) in Germany [1] which is publicly accessible. As such, the 
sensors and cameras can only be used conditionally to interact with the public, additionally 
the content must be legally admissible or negligible [2]. The facade itself consists of several 
components on the north and south facades. 


On the north facade of the building a LED Grid with 288x96 Pixel of RGB LEDs is integrated 
with the following dimensions: 7,62m x 2,54m along with, 36 Ambient Lights between the 
floors an the back projection with 12 Full HD projectors (8 frontal and 4 flanking) over the 
windows of the first and second floor. The south facade includes a back projection with 6 
projectors on the second floor.[3] Table 1 shows summarises the above data on the collective 
components of the media facade. 


The distance to the nearest campus building, to in north, is 33,5m. In between there is a less 
travelled road, parking spaces and a small place stretch of pebbles. At the location all German 
mobile communication providers offers the mobile communication standard LTE with up to 
150Mbit/s. 
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Components Ambient Light | LED-Grid Back Projection 
Number 36 1 18 
Resolution in Pixel 1x1 288x96 920 x 1080 
Viewing distance in m - min. 20-24 ~5 
Vertical DPI ~0,03 ~1,02 ~15,7 
Illumination in lumen 5000 5000 <3600 
Viewing angle 90° 120x120° 120° 
Operation time Night Day/Night Night 
Table 1: Visualisation properties of FKI-Facade 
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Figure 1: Media Facade FKI (northern front) Figure 2 View 


The existing architecture consists of four components: A client application receives the input 
data. The host application is responsible for displaying the content on the media facade. 
Figure 2 shows the architecture of the HTW Berlin media facade. 


Media Facade 


Launcher Host-Appllikctionen 


<<atifach> D 
Application 1 

««artifact»» D 
Application X 


Visualisation 


<<artifact>> D 
LED-Grid 


<<artifac>> D 
Back Projection 


Client-Application X 


Figure 3: Overview structure applications 


The Communicator component of the server facilitates the communication between the client 
and the host application. The Launcher component of the server is the central control unit to 
stop and start applications on the media facade, with a loop that waits for additional 
applications and a GUI for external selection of applications [4]. To visualise the content on 
the media facade the use of seven computers is required, of which each computer records one 
(sub)component. Starting an application causes the start of several smaller host applications 
for sub components (LED Grid and back projection) in the background. 
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3 EXISTING COLLABORATIVE INTERACTIONS STRATEGIES 


As an analysis of the interaction strategy of various collaborative applications they were 
tested in advanced t. The development of collaborative applications for media facades has 
been on going for the last 10 years but mainly incorporated in festivals and special events. 
The following briefly discusses the interaction strategies of various applications: 


One such example is the 2012 beating Valentine's Day heart in New York[10]. Through the 
use of a touchscreen stele, located on one side of the transparent cube encompassing a 
glowing heart, the heart could be made to pulse to different frequencies depending on the 
amount of hands touching the touch screen. 


The Shadow Play [11] application on the media facade at the HTW Berlin, allowed the users 
to determine the course of the next screen by majority. MegaPhone [12, 15], presented for the 
first time as a multi-user platform at the Urban screen Manchester 2007. One year later a 
collaborative version was presented to the NBA All-Star game in an Adidas store. As many as 
100 collaborators could simultaneously log in through their cell phone number, hence 
controlling a character through the pushing of phone keys. Through targeted shots at the 
opposing team one of the three stripes, each coloured differently, on the opponents jersey is 
removed. Two people simultaneously controlled Tetris stones in the 2010 Lummo Tetris 

[12] application, in the process they were tracked by a camera. One player defines the position 
of the stone and the other, the direction by moving left or right on the bottom of the displayed 
area. Action Flocking [5] tracked the movement of multiple people and created a graphic in 
which all the movements were displayed together. The company Inwindow Outdoor [17] 
developed a variety of tracking touch and gesture-camera applications for shop windows 
(usually by wiping or catching with hands). 


4 DIFFERENTIATION OF COLLABORATIVE APPLICATIONS 


Letellier divided collaborative applications on media facades into groups for communication, 
and productive applications and games. [14] According to Pinelle et al. [16] the division 
criteria can also be former after a a closely and loosely linked collaboration system. 
Consequently the choice of strategy or the interactions of the collaborative work depends on 
the chosen collaborative division. 


4.1 Closley Linked Collaboration 


To make collaboratively decisions for a task, continuous communication with each other has 
to take place. As a result there is a high dependency and interaction among the people in the 
group, thus they must be in spatial proximity and can only work synchronously. This is called 
a closely linked collaboration and is often found in games. The work on distributed media 
facades is only possible if they all synchronously display the same information and they are 
all located in the same media facades are in the same site. 


4.2 Loosley Linked Collaboration 


The various tasks to achieve the common goal can be delegated to different people within the 
group. Each person can edit their (sub)task independently of the other people in the group. 
Therefore the degree of communication and integration between persons within the group is 
low, as an example it could solely occur via email. Loosely linked collaboration is closely 
related with asynchronous work, as it is the case with Etherpad or OpenSteetMaps. However 
these applications require filtering and continuous (manual) control to avoid illegal content. 
This in turn leads to an increase in processing and as such in employment costs and time 
including a fault tolerance. 
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5 CRITERIA TO WORK COLLABORATIVELY 


The criteria for collaborative work can be divided into general criteria and criteria of media 
facades. Depending on the (sub-) tasks and the overall goal and the related it isn't always 
possible to regard or apply all criteria for collaboration. 


5.1 General criteria to work collaboratively 


* Time and location e Software and hardware use * Login 
* Usability (esp. language and * Transparency * Target 
cultural circle) * Motivation * Rules 
* Accessibility * Performance evaluation e Workspace 
* Platform independence * Working on content * Licenses/costs 
* user permissions * Communication * Number of users 


5.2 Criteria to work collaboratively on Media Facades 


* Operating life * Participation of collaborators 

* Request of collaboration * Size, type and position of Media Facade 
* Reaction time * Local conditions 

* Connection between collaborators * Distance to Media Facade 


* Distance of collaborators to each other 

The above criteria serve as a basis for further investigation into different strategies of 
collaborative applications and the development of appropriate approaches for collaborative 
interaction on media facades all over the world. 


6 BEST PRACTICE EXAMPLES HORSE RACE & PONG 
6.1 Game Idea 


The concept for mobile interaction was implemented for two exiting server-client 
applications, Pong and Horse Race. The implementation was to take place on the LED-Grid 
of the media facade with smartphones and tablets as input devices. Through this, the 
comparability of the research results were better comparable was increased. 


Pong is based on the classic game. Two opposing teams, each team controlling a paddle 
designed to prevent the ball from penetrate ones space. If the opposing teams ball manages to 
get passed ones paddle they score a point. Horse Race was developed as a front projection 
with collaborative interaction and four racecourses for the Festival of Lights, then fitted for 
the LED-Grid. The horses are on the left side of the LED grid. Through the shaking of the 
mobile device the horse can run to the goal on the right side. 


Figure 4: Existing Host Application Horse Race (LED-Grid) and Client Application 
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6.2 Implementation 


For the investigation a server-client application should be implemented, hence a host and 
client application have been developed. The host application used the LED grid and the back 
projection of the media facades. As a result, the area for the collaboration increased. For each 
component, a separate host application was developed, whereby the LED grid server 
application supplies the client application. This also received the input data from the client 
and forwards it to the Horse Race application on the computer of the back projection. Every 
host application analyses the data individually. A different solution had to be found for Pong, 
as the position of the ball and the movement direction can't be predicted yet they should still 
be decided in real time. Therefore the host application of the server of the LED grid renders 
the position oft the ball and the paddle every 6 seconds, then it sends the results to the 
computer of the back projection. 


6.3 Interaction 


A web application was implemented, which was accessible via a short URL, even from a 
mobile phone. In addition to a simplified and low-barrier dialog control the data was 
transmitted in real time. One of the goals of the development was to keep the GUI dialogs of 
web application straightforward and intuitive. All the information and elements required to 
participate in the test, were mapped onto the media facade and instructions on the possible 
interaction techniques where displayed on the mobile devices. The possible input interactions 
for Pong included the use of a touchscreen to swipe and select as well as the tilting and 
rotating of the device. The input interaction needed for Horse Race, was exclusively the 
shaking of the device. Thereupon it is important to note that each input interaction had a 
distinct response time for users. Thus the delay from input to reaction within the game 
diverges per game as the input methods differ. 


6.4 Collaborative Elements 


To support collaboration, different collaborative elements were implemented in the 
applications. The media facade is thus viewed as the main component, in which all 
information for the collaboration must be shown. The host application was therefore divided 
into four areas: workspace, login, information about the application, the goal and the team as 
well as the interaction. To compensate between the strength of the teams and to evaluate 
between the different forms of interaction, a unique login was limited to five 
collaborators/players for each team. Each collaborator/player is then assigned a unique ID, 
which he/she retains throughout the entire game, this serves as a facilitator for orientation and 
communication as well as the displaying of the game statistics. The statistics are displayed in 
real time throughout the interaction. Each application has its individual and unique 
implementation of the above. In Pong the selection of every collaborator was displayed and a 
team statistic in Horse Race, which shows how many shakes are necessary to win the race and 
the amount the team and every member currently possess. The length of the paddle was 
dependent on the number of collaborators in team. (1 collaborator — long paddle, 5 
collaborators short paddle) 


6.5 Algorithms 


For the evaluation of the input data, different algorithms for collaborative work were 
examined. The algorithm is based on whether a synchronous and / or asynchronous 
collaboration takes place (e.g. bundled or every value), the time slot of the input data and the 
number of logged (active and passive) collaborators. 


The Majority algorithm not only solved the simple or absolute majority determination (n/m), 
but also the number of choices and how many collaborators are connected. 

The arithmetic mean is often used in a conjunction with other algorithms. In the 
determination of the result (x....+...x,)/n can enter a standoff. Weighting the individual values 
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of the input data or choosing a random decision can result in a decision. A further possibility 
to extend the evaluation process is by dropping extreme values. As such, the passive 
participation can not be used to calculate the score negatively for the collaborative work. 
Heinrich describes that pessimistic algorithm can be used for collaborative work on the same 
document as Etherpad. [6,8] These are either turn-based, the write token moves from client to 
client, or blocking based, where a document is locked by a client with write access until the 
change is saved. A central authority manages the writing permissions. This makes them 
unsuitable for real-time communication and a synchronous collaboration. 

Optimistic algorithms (e.g. OT) - GOTO or COT algorithm) allow dedicated work on 
document copies that are continuously synchronised. [9] Therefore no special permissions are 
necessary. The synchronisation occurs on a operation based level, thus only the operations 
within the document are considered, i.e. the operations of individual characters will be 
recorded. In contrast, condition-based, the difference is calculated, for example through the 
OT algorithm that is used for Google Docs. Here a real-time collaboration is possible. In the 
case of incomprehensible conflicts arising, which cannot be solved through reconstruction, a 
protocol with which messages are exchanges is missing. 


"Scalable" algorithms for collaborative interaction are currently limited and only a few are 
available. Heinrich researched this topic in his dissertation [6,7]. For each application a 
individual scalable solution was implemented. The score in horse race was calculated with the 
number of collaborators form 1 till 5 in each team and the shakes of interaction of every team 
member. The calculation in Pong was to look for up and down and find the majority of 
direction every 10 seconds. 


7 EVALUATION 


The evaluation contained a user test, a stress test and a latency test. The tests where done for 
Android from 4.1, Windows 8.1 and 1OS systems with 16 different mobile devices on back- 
projection and LED-Grid. The applications were assessed for each 15 Minutes each. All users 
can connect to the URL with any browser available on the mobile device of their choice. They 
input data should be sent by mobile communication followed by WLAN. 


It was found that one of the main challenges was the synchronised mapping of all processes 
and implementation of the appropriate algorithms, due to the latencies on the projection 
devices which averaged between 100 ms and 200 ms, depending on the use of mobile 
communication or WLAN. 


The reference value is 100 ms, anything above will be a noticeable delay for the user. After 
the evaluation of the results, 66.7 % of the users were able to transmit wireless data to the 
server of the LED grid within the desired time slot of — 100ms. In mobile communications, 
the desired reference value was achieved, at least for the clients that use the LTE standard. To 
avoid excessive traffic load, the number of logged-in clients can be limited. Sending the data 
can occur in bundles of time slots of 10 ms to 100 ms. Table 2 and 3 show the results of the 
stress tests with the collaborators for wireless and mobile. 


Client | Mittel in Maximum in ms Minimum in ms 
1 73,1 2055,5 2,0 
3 86,4 777,5 3,0 
5 264,7 2811,5 6,0 
6 103,8 1206,5 8,0 
7 192,2 1381,0 5,0 
8 4119,1 29434,5 1,0 
9 18,7 344,0 2,0 
10 13,2 139,5 2,0 
11 23,6 993,0 2,0 


Table 2: Results evaluation WLAN 
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Client | Mittel in ms | Maximal in ms | Minimum in ms | Mobilfunkstandard 
1 208,2 2909.0 30,0 H+ 
2 634.6 6604.5 33,0 H+, E, G 
3 142,9 1015,5 40,0 H+ 
4 695,2 9868,5 29,0 H+ 
5 342,0 1388.0 40,5 H+ 
6 7607,2 18944,5 163,5 E+ 
7 192,2 1381,0 5,0 H+ 
8 490,0 13939,5 38,0 3G 
9 63,5 704,5 33,5 H+ 
10 111.9 329,0 41,0 H+ 
11 34,5 176,5 27,0 LTE 
12 42,6 499,5 3,0 LTE 
Table 3: Results evaluation mobile 
8 DEMONSTRATION OF THE DEVELOPMENT SYSTEMS 


These research projects collaborative applications Horse Race and Pong have been added to 
the applications of the media facade at the Research and Training Centre for Culture and 
Computer Science and were tested for the LED-Grid and back projection. This section 
contains demonstrative images of the executed Horse Race application on the media facade 
and the mobile device GUI. 
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Figure 5: (left) Media Facade - Horse Race, (right) GUI mobile device 
9 CONCLUSION AND FURTHER WORK 


First recommendations included the suggestion to set the focus exceedingly on the client 
applications and the GUI to expand all the information that is displayed on the media facade 
of the HTW Berlin. Future research focuses on the development of uniform scalable 
algorithms for the evaluation of the collaborative work forms of media facades and the usage 
behaviour of users. 


Currently the interaction access is limited, meaning only a certain number of users, as a result 
of the data load. A further investigation would be the optimisation of distributed systems to 
minimise latency and to give an unlimited number of users access to interact. 


With further work the GUI of the client may be integrated in other media facade applications. 
Another potential point of investigation is the dynamic allocation of work areas on the media 
facade. The challenge presents itself in the size and resolution of the various media facade 
components and their individual properties. 
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Abstract — FTV (Free-viewpoint Television) is the ultimate 3DTV with an infinite number of 
views and ranks as the top of visual media. It enables users to view 3D scenes by freely changing 
the viewpoint. MPEG has been developing FTV standards since 2001. In the first phase of FTV, 
MPEG developed MVC (Multiview Video Coding) standard, which enables efficient coding of 
multiview video. In the second phase of FTV, MPEG developed 3DV (3D Video) standard for 
multiview displays. Based on recent development of 3D technology, MPEG has started the third 
phase of FTV, targeting super multiview and free navigation applications. This new FTV 
standardization will revolutionize 3D viewing of the scene. Users can enjoy very realistic 3D 
viewing and walk-through/fly-through experience of 3D scenes. 


INTRODUCTION 


4k/8k UHDTV (Ultra High-Definition TV) has realized viewing at the highest resolution in 
visual media. However, it transmits only a single view and users can’t change the viewpoints. 
It is still far away from our viewing experience in the real world. 

The next challenge is FTV (Free-viewpoint TV) [1]-[7]. FTV enables users to view a scene 
by freely changing the viewpoints as we do naturally in the real world. FTV is the ultimate 
3DTV with an infinite number of views and ranks as the top of visual media. It is also the best 
interface between human and environment, and an innovative tool to create new types of 
content and art. FTV will give a great impact on the various fields of our life and society. 

MPEG has been developing FTV standards since 2001. MVC (Multi-view Video Coding) 
[8] was the first phase of FTV, which enables the efficient coding of multiple camera views. 
3DV (3D Video) [9] is the second phase of FTV, which enables viewing adaptation and 
display adaptation of multiview displays. Based on recent development of 3D technology, 
MPEG started the third phase of FTV [10] in August 2013, targeting super multiview and free 
navigation applications. The vision of this third phase is to establish a new FTV framework 
that revolutionizes the viewing of 3D scenes. 


HISTORY OF FTV STANDARDIZATION IN MPEG 


MPEG has been developing FTV standards since 2001. Fig. 1 shows the history of FTV 
standardization in MPEG. In 2001, FTV was proposed to MPEG and the 3DAV (3D Audio 
Visual) activity started. In the 3DAV activity, many 3D topics such as omnidirectional video, 
FTV, stereoscopic video and 3DTV with depth disparity information were discussed. 
According to the result of Call for Comments from the industry, the discussion was converged 
on FTV and MVC (Multi-view Video Coding) [8] started in March 2004. 

MVC is a coding standard of multiple video. MVC is the first phase of FTV. The MVC 
activity moved to the Joint Video Team (JVT) of MPEG and ITU-T for further 
standardization processes in July 2006. MVC was completed in March 2009. 

MPEG started 3DV (3D Video) as the second phase of FTV in April 2007. 3DV is a 
standard that targets serving for a variety of multi-view 3D displays [9]. Although the number 
of input views and that of output views are the same in MVC, the number of output views is 
larger than that of input views in 3DV. View synthesis is introduced into 3DV to increase the 
number of views. The 3DV activity moved to the Joint Collaborative Team JCT-3V for 
further standardization processes in July 2012. The 3DV will be completed this year. 
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In August 2013, the third phase of FTV started to revolutionize 3D viewing by super 
multiview and free navigation [10]. 
MVC, 3DV and FTV standardization will be described in more detail in the following. 


2001/ 2002/  2003/  2004/ 2005/ 2006/ 2007/ 2011  2011/ 2016/ 2020 . 
2 12 10 10 07 01 07 03 11 02 me, 
FTV 3DAV ||CfC on |CfE CfP Evaluation ||Req. on | CfP Evaluation of} |CfP 
Proposed |seminar|3DAV  |on MVC|on MVC|of proposals||FTV on 3DV ||proposals on FTV 
2001/12 , | 
Start 3DAV i | 
: 2004/03 i ! 
3DAV ! Start MVC (Multi-view | 2009/05 
«Requirements on ' Video Coding) ! MVC completed i 
DAV. || MVC (moved to JVT in 2006/07) i 
*EEs on 3DAV | l MV-HEVC 
Targets | a | 2007/04 MVC+D | 
converged: ee i Sart a DN, 3D-AVC | 
onFTV | | (3D Video) — 3D-HEVC! 


3DV (moved to JCT-3V in 2012/07) 


| : 2009/02 


| | À ! Revolutioni 

i ! Requirements on : 2013/08 NaN e. 

i ! 3DV ! Start NEAR LES 
:ETV 

| | | FTV 

i rie NP 

! First phase of FTV i Second phase of FTV — : Third phase of FTV 


Fig. 1. History of FTV standardization in MPEG. 


MVC AND 3DV STANDARDIZATION 


MVC targeted efficient coding of multi-view video. In MVC, the number of input views is 
the same as that of output views. The view synthesis function of FTV is not included in MVC. 
MVC was standardized as the extension of H.264/MPEG4-AVC [11]. MVC standard has 
been adopted by Blu-ray 3D. 

View synthesis is introduced into 3DV. 3DV sends small number of views and generates 
large number of views at the receiver side for multiview displays. Multi-view and multi-depth 
are sent to the receiver side and the intermediate views are synthesized from views with 
assistance of depth information at the receiver side. 3DV enables display adaptation and 
viewing adaptation [12]. 

Although MVC and 3DV standards provide useful tools for multiview-3DTV, they don't 
support full FTV functions. View synthesis is not considered in MVC. Number of views to be 
encoded is limited to less than 63 in 3DV. Furthermore, view synthesis is limited to 
interpolation along a horizontal base line of linear camera setup. These limitations have to be 
removed to introduce new FTV applications. For example, multiview displays provide glass- 
free 3D viewing with motion parallax. However, their motion parallax is not smooth and the 
viewer feels visual fatigue when viewing long. 

Realistic 3D viewing with smooth motion parallax and without visual fatigue needs super 
multiview displays [13] that are not supported by the MVC and 3DV standards. The reason is 
described below. 

Fig. 2 shows responses of eyes in real scene. As shown here, accommodation and 
vergence are matched because rays come from the object. 
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Thickness Object 
of lens 
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Rotation 


of eyeballs Vergence 


Two eyes look here 


Fig. 2. Responses of eyes in real scene. 


However, accommodation and vergence are not matched in artificial scene as shown in 
Fig. 3 because rays come from the screen. It is accommodation-vergence conflict and causes 


visual fatigue. 


Screen of 
3D display 


3D image 


mismá 
Two eyes|look here 
Vergence Lens focuses here 


Accommodation 


Fig. 3. Responses of eyes in artificial scene. 


The accommodation-vergence conflict can be avoided by dense rays satisfying super 
multiview condition as shown in Fig. 4. Here, the following feature of eye is used: When two 
or more rays passing through the same point enter the pupil simultaneously, the eye can focus 
on that point. Super multiview with view interval S1 degree is needed for convergent case to 


satisfy this condition. 


Accommodation 


Viewpoints 


Screen 
Fig. 4. Super multiview condition to avoid accommodation-vergence conflict. 


FTV STANDARDIZATION 


Framework of FTV 
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In 2010, 2022 FIFA World Cup Japan Bid Committee planned to deliver the excitement 
on soccer stadium to the world by FTV. It aimed to revolutionize the viewing of the soccer 
game by super multiview and free navigation as shown in Fig. 5. Super multiview realizes 
realistic 3D viewing of the scene and free navigation realizes walk-through or fly-through 
experience of the scene. 


Super multiview Free navigation 


Realistic 3D viewing Walk-through or fly-through 
Fig. 5. Revolutionized viewing of 3D scene by FTV. 


According to such motivation, the framework of FTV is determined as shown in Fig. 6 
[14]. FTV has two types of outputs. One is super multiview that reproduces very high number 
and high density of views for super multiview displays. Another is a single view with freely 
changing viewpoint for free navigation. 


Display 


Capture 4 D 


FTV 
Encoder | Trans- 


Sender side Receiver side 


Free navigation 


Fig. 6. Framework of FTV. 
FTV Applications 


FTV has two types of applications [14], [15]. 
(1) Super multiview 


Fig. 7 shows a HoloVizio Light-field display, SOWLT. It displays 78 views each of which 
has 1024x768 pixels. It has very wide view range of nearly 180 degrees. The feature of super 
multiview displays is not only a large number of views but also high density of displayed 
views. Because of these features, users can see a 3D scene from any directions with smooth 
motion parallax. Fig. 7 shows examples of displayed images in different directions. 
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HOLOVIZIO”sowLT 
(a) left view (b) right view 


Fig. 7. HoloVizio Light-field display, SOWLT, and displayed images in different directions 
(Holografika). 


Fig. 8 shows displayed images of another type of super multiview display with 256 views 
in different directions [13]. 


(a) left view (b) right view 


Fig. 8. Displayed images of 256-view super multiview display in different directions (Tokyo 
University of Agriculture and Technology). 


Fig. 9 shows life size 3D images of a 200-inch 170-view display. 


(a) left view (b) right view 


Fig. 9. Life size views of the 200-inch 170-view display in different directions (NICT). 


Fig. 10 shows an Integral Photography (IP) system [16]. It has 400x250 views and 
provides not only horizontal parallax but also vertical parallax. 
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(a) lower view (b) upper view 
Fig. 10. Integral 3 DTV giving both horizontal and vertical parallaxes (400x250 views) 
(NHK). 


Fig. 11 shows a portable 360-degree viewable 3D display, *Holo-Table" [17]. It displays 
all-around 3D image floating on the table. It has 360 views each of which has 1024x768 
pixels. Fig. 12 shows examples of displayed images in different directions. 


3D View 
(Horizontal 
parallax only) 


Fig. 12. Displayed images of Holo-Table in different directions. 


(2) Free navigation 


Free navigation applications are considered as a potential new market for communication 
companies. Fig. 13 shows FTV on a mobile player. 
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Fig. 13. FTV on mobile devices (left: Nagoya University, right: Orange Labs Poland). 


Walk-Through or Fly-Through experience by FTV has been developed using *3D model 
plus texture mapping" [18]-[20]. This technology enables free viewpoint viewing of soccer 
games as shown in Fig. 14. 


Fig. 14. Free navigation viewing of soccer games (KDDI). 


CONCLUSION 


FTV is the ultimate 3DTV that transmits all visual information of a 3D scene and ranks as 
the top of visual media. It will find many applications in the fields of broadcast, 
communication, entertainment, advertising, design, exhibition, education, medicine, art, 
cultural heritage and so on. Super multiview and free navigation functions of FTV will 
revolutionize 3D viewing of the scene. MPEG has issued Draft Call for Evidence on FTV 
[21]. It is expected that FTV technologies will be standardized by 2020 and used in 2020 
Tokyo Olympics. 
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Abstract — FTV (Free-viewpoint Television) enables users to view 3D scenes by freely 
changing the viewpoint. MPEG has been developing FTV standards since 2001. The third phase 
of FTV started in August 2013. This new FTV standardization targets super multiview and free 
navigation applications. Free-viewpoint image generation from free setup cameras is needed for 
free navigation. Free-viewpoint images are successfully generated for static scenes from views 
captured by moving a single camera freely. This technology can be applied to free navigation 
application of FTV. 


INTRODUCTION 


FTV (Free-viewpoint Television) [1]-[7] enables users to view 3D scenes by freely 
changing the viewpoint. MPEG has been developing FTV standards since 2001. The third 
phase of FTV [8] started in August 2013. This new FTV standardization targets super 
multiview and free navigation applications [9]. Free-viewpoint image generation is needed for 
free navigation. Free-viewpoint image generation from regularly arranged cameras such as 
linear or arc camera setup has been studied. However, it is not easy to arrange cameras 
regularly in actual situation. Therefore, free-viewpoint image generation from free setup 
cameras is strongly requested. 


2 FREE-VIEWPOINT IMAGE GENERATION SCHEME 


We realized free navigation for static scenes by using a freely moving handheld camera. 
This technology is based on Structure from Motion and any special markers are not used [10]- 
[12]. This technology can be applied to free navigation for dynamic scenes using free setup 
cameras, too. The flow chart of free-viewpoint image generation is shown in Fig. 1. It consists 
of the following 6 steps. 


1. A scene is captured by a moving single handheld camera. Captured video is divided into 
frames and these frames are regarded as one frame of multiview. 

2. First, feature points are detected by using Scale Invariant Feature Transform (SIFT) [13] 
and then matched feature pairs are found in all views. Using the feature points and the 
matched information among views, we estimate extrinsic parameters (pose/position), 
intrinsic parameters (focal length, lens distortions) and 3D positions of the feature points 
by Structure from Motion “Bundler” [14]. 

3. Given virtual camera location, i.e. free-viewpoint image location, depth search range is 
estimated by reconstructed feature points. 

4. Some reference images are selected from views for estimating a depth map and generating 
a virtual view image. 

5. By using the selected reference images, a depth map is estimated at the location of the 
virtual camera. Block matching, graph cut algorithm and feature points are used to get a 
high quality depth map. 

6. The virtual image is rendered by using the estimated depth map and the selected reference 
images. 
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Capturing by a handy camera 
Camera Calibration 
Search Range Setting 


Lens Distortion Correction 


Feature Detection by SIFT 


3D Reconstruction by 
Structure From Motion 


Reference Images Selection 
Projective Block Matching 
Depth Estimation Depth Map Improvement 


Virtual Image Generation Graph Cut Algorithm 


Fig. 1 Flow chart of free-viewpoint image generation for static scenes. 


In free navigation for dynamic scenes, step 1 and step 2 are applied to free setup cameras. 
Then, step 3 to step 6 are applied to each virtual camera location. 


EXPERIMENTS 


Fig. 2 shows an example of captured view by moving a single camera and detected feature 
points. 


Handy Camera 


Fig. 2 Captured view by moving a single camera and detected feature points. 


Fig. 3 and Fig. 4 show captured video, generated free-viewpoint images and camera 
trajectories of indoor and outdoor scenes, respectively. The trajectory of detected actual 
camera movement is shown in blue and that of given virtual camera movement is shown in 
orange. 

Fig. 5 shows examples of generated image and depth map. 
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actual camera movement (blue) and 
virtual camera movement (orange) Generated Free-viewpoint Images 


Fig. 3 Captured video, generated free-viewpoint images and camera trajectories of indoor 
scene. 


ZZA 


Captured Video 


ICE BS E EN 


actual camera movement (blue) and 


: Generated Free-viewpoint Images 
virtual camera movement (orange) 


Fig. 4 Captured video, generated free-viewpoint images and camera trajectories of outdoor 
scene. 


(a) Generated image (b) Generated depth map 


Fig.5 Generated image and depth map. 


67 


CONCLUSION 


Free-viewpoint images are successfully generated for static scenes from views captured by 
moving a single camera freely. This technology can be applied to free navigation application 
of FTV. 
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Abstract 
This work provides insight into Russian cinema-goers relations with social media. The Internet 
adduces potential cinema-goers from watching movies in theaters, but the social networks in 
Internet creates new framework for theater audience development. The platforms VK and 
Facebook are most popular in this context. Some trends for how social media marketing 
influences movie theater business in metropolitan areas and small towns of Russia are exposed 
and discussed. 


INTRODUCTION 


At the end of 2013 Russia ranked 7th position in the Top 20 International Box Office 
Markets with total Box Office 1.4 US$ Billions [1]. 

As of January 1, 2014, there were 3479 commercial screens in 1,101 cinemas (the average 
number of screens per site is 3.2). 85% of modern cinemas in the territory of Russia feature 
digital equipment, which amounts to 2,967 screens in total (in 1,010 movie theaters, which is 
92%) 2,478 (84%) of which feature equipment for 3D screenings. The number of cinemas 
with a digital projector for each screen has reached 809 with 2,363 screens in them (74% of 
the total number of cinemas and 68% of the total number of screens) [2]. 

At the end of 2013 Russia is well on its way toward reaching its goal of providing Internet 
access to 93% of its population. Russia's mobile phone market is fairly saturated. More than 
nine in 10 Russians (91.5%) say they personally have mobile phones. Ownership is 
ubiquitous across both urban areas (93.896) and rural areas (86.896). Three in 10 mobile 
phone owners (29.6%) say their phones are smartphones, with the ability of accessing the 
Internet and downloading apps. Internet use and social networking are still the domain of the 
young in Russia. More than nine in 10 Russians aged 15 to 24 (92.1%) have used the Internet 
in the past week; this number falls to 31.3% among those 45 and older [3]. 

So, main cinema industry actors - studios, distributors and exhibitors — are working on 
almost completely digitized and web-communicated Russian landscape 

These conditions provide an excellent framework for enrichment of cinema marketing 
strategies. But, the same framework not means that the actors listed above have the same 
marketing goals. For the studio the movie is considered as fodder for television channels or 
internet on-line cinemas. Income by the film distribution on these platforms is added for the 
studio to the income from the movie screening in the world-wide cinema networks. On the 
other hand, each theater receives income from visitors (cinema-goers) who come to this 
theater. 


SOCIAL MEDIA AND CINEMA INDUSTRY 


Figure 1 shows the popularity of main social networks in Russia. First three are specific for 
Russia only, second three are world-wide media. The premiere social network is VK 
(Vkontakte), gets called the Russian clone of Facebook. VK include personal pages, photo 
sharing, and community groups for people, geared towards providing entertainment media, 
and the ability to find TV shows and films to watch on-line or in the cinema. VK users spend 
more time for listening music and watching shows than the standard social browsing done on 
Facebook. The popularity of VK and other local social media is due to their cultural 
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appropriateness not for only Russia, but for many other countries with Russian-speaking 
population. 
0 10 20 30 40 50 
VKontakte 
Odnoklassniki 
MoiMir 
Facebook 
Livejornal 


Twitter 
Source: TNS Web Index May 2014 


Figure 1. Monthly audience of main social networks in Russia, millions 


At the same time all actual international surveys show a significant predominance of 
Facebook and Twitter around the world, where VK is ranked at the end of ten. 


THE ISSUES 


Audience trends in Russian cinema markets are decreasing and here we need to develop 
new methods and new partnerships to attract younger audiences to the cinema. In addition, the 
Russian economy is in a stage of turbulence again and this process directly concerns the 
national film industry. The new ideas regarding social media, mobile communications and 
customer relationship management are developed by the leaders of market and by the each 
cinema theater. 

First, use of information sources that influence online purchase decisions strongly varies 
by culture. Cultural differences across countries with respect to products and services bought 
on the Internet appear to mirror differences found in traditional shopping channels [4]. The 
study [5] introduces different cultural dimensions to compare the use of social media and 
other information sources for consumer decision-making across 50 countries. The dimensions 
are: Individualism/collectivism, Uncertainty avoidance, Long-/short-term orientation and 
some others. But you can't find here any data about Russia. So, features and preferences of 
large national audience should be taken into consideration when promoting films both from 
Hollywood and national film production. Especially for Chinese motion picture industry the 
study [6] collects the empirical data and reveals word of mouth (WOM) and online WOM 
influence on movie going decision making. The global and local cinema marketers need more 
investigations about this issue. 

Second, WOM now is in the focus of interest among movies and cinema marketing 
practitioners and consumers. WOM itself is an ‘informal communications directed at other 
consumers about the ownership, usage, or characteristics of particular goods and services 
and/or their sellers" [7]. But, well-known strategies of marketing of any media entertainments 
in social media can’t be applied “as-is” for the movies and cinema industry. The promises of 
WOM marketing are often oversold, and various assertions about the nature of WOM, its 
dynamics, antecedents, and consequences at times have been misstated in mass-mediated 
articles and books on the topic [8]. Thus, it was found that potential moviegoers reacted more 
favorably to seeing a movie in theaters when viewing the official movie website as opposed to 
the social media page. Movie marketers should focus on using social media pages to 
encourage people to visit the official website in order to maximize interest in viewing the film 
in theaters [9]. 

Third, we need to clarify differences between marketing of any film for studio [10][9] and 
marketing of any cinema theater as itself. 

In this work we try to provide the short survey and analysis of Russian cinema-goers 
relations with social media. 
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WOM ABOUT CINEMA IN RUSSIAN INTERNET 


In Russian segment of Internet (called Runet) there are thousands of websites about the 
movies. Studios, distributors, cinema networks and other entities are very active in Runet and 
present in all popular social media platform, listed above. Vast majority of communication 
directed at the audience of *movie viewers" than movie goers. In situation when the each 
movie itself is only one part of wide spectrum of internet-available media entertainments, 
such activities lead the potential cinema-goers to find this movie in one click for seeing in 
internet and thus decline cinema going. 

More closer to cinema-goers is the national-wide resource www.kinopoisk.ru with a slogan 
“Find your movie". They have now about 1.5 millions in the VK group, 256K likes in 
Facebook, and 179K followers in Twitter. Visitors willing to vote in many on-line surveys, 
for example, more than 30.000 participate in voting “Which of the movie premier in April 
expecting most", where "Fast & Furious 7" wins with about 50% answers. 

Resource www.kinopoisk.ru is the one of the most viewed, when audience seeks trusted 
estimation of forthcoming movie releases. The survey [9] indicates the IMBD.com placed on 
the top for USA, while overwhelming majority of Russians does not even take it into account, 
together with official movie studio website. 

Internet resources about entertainments, such as http://www.afisha.ru/spb provide for 
cinema visitors place to leave reviews and opinions about movies and cinema theaters. 
Opinions are often contradictory: *Convenient system to call waiter right to seat" and “The 
only thing that interferes with - this is when someone calls the waiter". Even so, in 
distinguishing between high- and low-quality cinema theaters before going, potential 
moviegoers look for credible signals from previous cinema visitors. This is a clear example of 
WOM applied for cinemas. Due to the fact that practically each user leaves here their contacts 
in social networks, they can to continue interpersonal WOM-communication about other 
topics. In framework of social media optimization (SOM) user must be redirected to the web- 
site of the cinema theater. But no, in the vast number of cases, you will be taken to the site of 
cinema network which own the cinema theater. So, we need to look closely on cinema 
networks. 


Cinemas in Metropolitan Areas 


In Russian Federation there are 15 big cities with population more than 1 million. The 
major players here are all-state Russian cinema networks which own most of theaters. Look at 
the situation in St.-Petersburg with a population of more than 5 million, right after Moscow. 

By March 2015 total amount of cinema theaters in St.-Petersburg is about 60 as reported 
by tickets sales services. Most of them — 51 - belong to cinema network, as shown in Table 1. 
These theaters haven't own websites or pages/accounts in the main social networks, in other 
words the “face” of each theater is hidden by network. If someone wants to learn more about 
a particular movie theater of this group, he is redirected in a SOM strategy to the web-site of 
cinema network. At this page you can find links to pages/accounts in social networks, 
associated with network again, not specific theater. 

It is obvious from the Table 1 that social media VK, Twitter and Facebook combined for 
most widespread networks. Platforms Foursquare, Google+, Livejournal, Odnorlassniki not 
listed because are linked no more than two times from the cinema networks. 

There is a clear correlation between the amounts of theaters and amounts of pages/accounts 
in social networks belongings each cinema network. A big question is what is primary and 
what is secondary, as in the example with chicken and egg. No doubts, to bring its owner 
golden eggs, cinemas must present in social networks. 

The only exception is the owned by state non-commercial network *Petersburg-cinema" 
(www.peterburg-kino.spb.ru). This is practically the only cinema network in the Russia, 
preserving children's theaters in state ownership, and developing them as multifunctional 
cultural and educational centers. 
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Cinema Network Social Network 
5 Total 
Total E 4 g 2 Accounts in 
WEB Site Cinemas in | 5 A |8 b | È Social 
St.-Petersburg) J S * $18 Network 
> m |E |S |> 

www.formulakino.ru 12 Y Y Y Y Y 5 
www.mirage.ru 9 Y Y Y Y Y 5 
www.karofilm.ru 8 Y Y Y Y 4 
www.kino-5d.ru 7 Y Y|Y 3 
www.peterburg-kino.spb.ru 6 Y 1 
www.cinemapark.ru 3 Y Y Y|Y Y 5 
www.luxorfilm.ru 2 Y Y 4 3 
www.cinemastar.ru 1 Y Y Y 3 
http://kino-city.ru/ 1 Y Y Y 3 
www.mori-cinema.ru 1 Y Y 2 
www.kinopolis-film.ru 1 Y 1 
Total 51 10 9 8 5 3 35 


Table 1. St.-Petersburg Networked Cinemas in Social Networks 


Methods of work with the audience are not particularly different from those generally 
accepted in other sectors of the entertainment industry. Marketers continue to view social 
media as just another place to post advertisements. The audience sensitively captures when 
media publicity is generated by paid media and vice versa, WOM as user-generated content is 
usually perceived as more credible and trustworthy. Only a small number of real cinema- 
goers are present in the lists of participants or followers of cinema networks. Pages are 
crowded by official posts from studios, random members or someone’s who want to sell 
anything. 

Table 2 shows the key numerical characteristics reflecting cinema networks activity in two 
most popular social media — Vkontakte and Facebook. 


Cinema Network Social Network 

WEB Site vk.com/ Pts* Facebook/ Likes 
www.formulakino.ru formula kino 23801 | formulakino 6635 
www.mirage.ru miragecinema 98211 | miragecinema 499 
www.karofilm.ru karofilm vk 22585 | karofilm 6124 
www.kino-5d.ru club25231150 5300 | pages/Kino-5D 13 
www.cinemapark.ru cinemapark.russia | 16817 | cinemapark.russia 1770 
www.luxorfilm.ru luxorofficial 90427 | luxor.official 5969 
www.cinemastar.ru cinemastar 38664 | cinemastarofficial 11009 
http://kino-city.ru/ kinocenterkinocity | 2428 | KinoCity.Novosib 500 
www.mori-cinema.ru - - MORI-Cinema 85 
www.kinopolis-film.ru kinopolisubiley 5644 - - 


Table 2. Cinema networks in the main social networks 
* Participants in official group only 


“Like” in Facebook is a good indicator of interest in cinemas of the network. A 
participation in the VK group shows a raised level of awareness about the cinema network at 
whole. 
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Within the time of next crisis in national economy the cinema networks focus their efforts 
on the outermost layer of the population. For poor — tickets with lowest price to morning, for 
rich — individual cinema show with champagne in time on your choice. 

This is realism than population of small towns in Russia has more equal income, so 
cinemas located here must to be more creative in development of their audience. 


Cinemas in Small Towns 


Cinema theaters in Russia are unevenly distributed between metropolitan areas and small 
towns. One third situated in Moscow and Saint-Petersburg, where only 16% of population. 
About 30% live in small towns, where are 10% of cinemas (Source: Nevafilm Research 
www.nevafilm.ru). However, activities of small town cinemas in social media are very 
interesting for analysis. Look at the situation in St.-Petersburg region around city of St.- 
Petersburg. 9 towns in the region have more than 50.000 of people, each of town has one or 
more cinemas. 

Table 3 shows how using social networking in a constant marketing campaign allows 
theaters in these small towns to establish a two-way communications with prospective 
cinema-goers. 


Cinema | Web | Town | Population | Cinema Items for Participants | % of Pts/ 
# Site Owner | discussion | in group Population 
1 Tikhvin 58000 Private 51 13755 24 
2 Kirishi 52000 Private 14 8390 16 
3 Y | Gatchina 95000 Private 3l 12278 13 
4 Y | Volkhov 46000 Private 3 4976 11 
3 Volkhov 46000 State 0 3895 8.5 
6 Y | Gatchina 95000 Private 0 3857 4.1 
7 Y Kolpino 140000 State 7 824 0.6 
8 Pushkin 93000 State 1 459 0.5 

Table 3. Ranked list of cinemas in small towns of St.-Petersburg region, based on VK 
activities 


This is interesting to compare statistics in Table 3 and Table 2. Cinema theater £1 in 
Tikhvin has practically the same amount of participants in social network VK, as an all-state 
cinema network “Cinemaprk”, operating in 15+ towns with population 80+ millions. Only 
half of cinemas in small towns hold own web-site, this is clear pointer for strategy of social 
media marketing (SOM), when main vector of activities is directed to promotion in social 
networks only. Three cinemas in Table 3 are state-owned, others are private. Doings of first 
reflects rather social responsibility contradictory for seconds, which are completely business- 
driven entities. 

For cinema in small town is vitally to create and maintain a group of followers in social 
media. It is not needed to depend upon dream that the members of group will create a 
coherent logical structure and interesting content for the page of communication. The 
members of group must constantly feel that opinion of group at whole and every member 
individually is very important for the cinema. If participants of group sense a care from 
theater, they are more likely bring their friends and pay to see a movie in this theater. Many 
meticulous users look attentively, how often information is renovated on the pages of social 
networks. So some owners of theaters specify for staff the time delay for answer to user's 
comments, for example - no more than three hours. The best results may be illustrated by 
results of voting *How often are you going to the cinema?" at Figure 2. 
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1perweek (I 31% 
1permonh [55% 


1peryear m 5% 
Only with children ga 3% 
No, only Internet E 6% 


Source: Pilot Cinema, Gatchina. Retrieved from http://vk.com/board46047 162 05 March 2015 
Figure 2. Group Voting “How often are you going to the cinema?" 


The programs of loyalty must be conducted constantly, and the results of competitions and 
lotteries need to be published immediately, to save intrigue of communication both inside the 
group and in group communication with cinema administration. 

Online surveys are vitally important to cinema theaters to disseminate information about 
their services. Most predominated topics of surveys are: “Which old movies you want to book 
for group seeing", *Are you want to participate in non-stop night themed show", *Are you 
want to go to the alternative content show" etc. 

Figure 3 shows results of one of such survey. See as traditional “face-to-face” WOM 


among family, friends, and others is absorbed by electronic word of mouth (e-WOM). 


E-mail EJ 996 

Newspaper B 396 

Street Billboards [n 1396 
Cinema Billboards SE 24% 
Mailbox Delivery TEM 8% 
Flyers at public places EH 10% 
Information for group (MN 33% 


Source: KinoTikhvin. Retrieved from http://vk.com/board31406141 05 March 2015 
Figure 3. Group Voting “Preferred information source about current films” 


Goings-on of cinema in social media make the group a major source of information about 
movie releases. The volume of WOM has an important effect on awareness of a movie. When 
the volume increases, moviegoers are more prone toward expecting an upcoming movie to be 
good. 

Records from users provide for cinema more clear understanding of his audience. One of 
the questions still unforgettable: “Does it real, that each cinema-goer gets individual 3D- 
glasses?". Continuing the topic, we should mention as one theater in small town held a contest 
where the grand prize was the personal 3-D glasses. It was probably a stroke of genius, if only 
this cinema was compatible with this prize. 

Popular topics are discussed more frequently than less popular ones through positive, 
negative, or neutral WOM messages. Cinema-goers are exposed to them repeatedly, the 
exposure effect could have a significant impact on maintain good reputation and start to 
improve service and equipment in the cinema. 


CONCLUSION 

Different kinds of internet-resources related with the movie industry, social networking, 
and marketing using social networking were examined to better understand the Russian 
context. The survey confirms the widespread use of social media by cinema-goers, 
particularly VK and Facebook. The effectiveness of the marketing almost entirely hinges on 
the features of the cinema theater activities in communication with potential and real cinema- 
goers. With more competition from other entertainments, cinema theaters need to improve 
their methods of attracting audiences, especially young. 


The work did expose trends for how social media marketing influences movie theater 
business in metropolitan areas and small towns in Russia. With decreasing attendance, cinema 
theaters should improve current and contrive new online marketing methods for most impact 
on target audiences. 

Now the cinema motto must to be “We sell because we are friends" as opposite “We are 
friends because we sell". 
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Abstract 

Screenwriting and film editing have much in common: both are essentially iterative and mending 
crafts. Improvement comes through failure. Editors use an interface that allows for rapid ‘failure’ and 
so arrive at a good cut fairly quickly. By contrast the tools and processes that screenwriters use tend 
towards slow failure; they're effectively still using a typewriter. 

This paper looks at the craft of film editing and the tools they use and asks: in this world of digital 
plenty, to what degree can screenwriters prototype their stories for the big screen? 


PROVING THE STORY 
This is a true story about one feature film. I know the director and I’ve interviewed both of 
the film editors. Two editors... Is that one too many? 


The film is called Kill Time (I’ve changed the name to protect the innocent). It's by an A-list 
screenwriter. Everyone who's read it thinks it's wonderful. The film is green lit; the budget is 
set at $80 million. 


There's a four-month shoot; all goes well. Then the director and all the film rushes fetch up 
in a small dark room with an editor. The editor begins to cut the material and, after a month 
or so, he quietly confides to his wife one evening: 'the story doesn't work”. 


There's a test-screening at a New York cinema; the audience finds the story confusing and 
gives it one of the lowest ratings ever recorded. The Exec Producer orders several re-cuts, 
and then hires a new film editor. Another director is drafted in to help the first director. They 
change about 40% of the original story, and spend a month on reshoots. At the next test 
screening, the new film fares only slightly better than the first. The film goes straight to 
DVD where it's given a ferocious mauling by the critics. It's taken four years to complete, 
the final budget exceeds $100 million dollars, the film is poor, and everyone's reputation is in 
tatters. 


The story doesn't work? How can this be? The screenplay was a triumph on the printed 
page. But in the translation to the screen it fell apart. Perhaps we should not be too 
surprised; word and moving image are two discrete mediums, and the story ‘physics’ of each 
are quite different. When you're reading a script, you're clearly not seeing the film. But Kill 
Time, like many well-written scripts, can seduce us into believing we really are seeing the 
film. 


How do we make our scripts fit to flourish and survive the journey ahead? The shoot is an 
extreme environment: direction, acting, wardrobe, choreography, cinematography, location, 
lighting, and the vagaries of the weather; the script must anticipate the ‘violence’ of this 
phase. And it must also anticipate the phase beyond. Editing is a beautiful, alchemical craft. 
But it is also a kind of lie detector; and the biggest lie it detects 1s the flaw in the story. 


A script, which is a simple word document, may attract millions in investment. And yet this 
investment is made with no clear proof that the story will survive the exigencies of the shoot 
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and the edit. Those test screenings of Kil] Time in New York - they took place after the 
money was spent. Too late. 


"Films originate from written words. Words here, now; images there, later. The question 
that has plagued the motion pictures almost from the beginning is how best to proceed from 
one medium to the next." Jean Pierre Geuens [1] 


This paper looks at the relationship between screenwriting and film editing, and asks: in this 
world of digital plenty, to what degree can we prototype our stories for the big screen? 


"The humbling truth is that the film is made in the editing room." David Mamet [2] 
"Editing is what allowed films to take flight" Walter Murch [3] 
"Editing is what makes a movie a movie" Zach Staenberg [4] 


Editors are the final ‘writers’ of the script. But of course editors would never claim to be 
authors; and they rarely work alone. In this paper, when I talk of editing, it should be 
understood I’m referring to that special creative relationship between director and editor. 


In the world of film, writing and film editing are almost entirely detached crafts; it is quite 
rare to find a writer in the edit suite. Yet the writer and the editor have much in common. 
The screenwriter is a story-inventor, and the editor is a raconteur. The one invents a story 
from a world of almost infinite possibilities, and then expresses it in the medium of words. 
The other constructs the best telling of that story in the medium of time, image, sound and 
motion. 


One might expect screenwriters to take a lively interest in how editors tell their stories. But if 
one looks at the canon - Field, McKee, Snyder, Vogler, and Truby, and also the screenwriting 
communities such as Triggerstreet.com - there are surprisingly few references to film editing, 

or to images. 


Why is this? For reasons we will discuss later, there is a longstanding injunction on 
screenwriters to avoid writing in shots, or to use audio descriptors, or indeed anything that 
prefigures the shoot and the edit. The *how-to gurus' have mostly upheld this convention, and 
this has tended to promote a non-technical appreciation of the medium for which we are 
writing. Thus as screenwriters, when we think of film, we are more likely to think of 
something projected rather than edited; in other words film as an already-finished product; 
something that unfolds as an 'invisibly edited” flow of sound and pictures. 


But this paper suggests that as screenwriters, to truly appreciate the ‘physics’ of cinema, we 
need to break the surface of the finished film, and in a sense make the cut visible. 


I'm undertaking a PhD by practice, and the practice element is a screenplay. 

I want to write in a way that anticipates the storytelling of the edit suite. I call this ‘writing 
for the cut'. The research element is a quest to discover how editors tell stories, and whether 
there are fresh insights that can be passed back to the screenwriter. I'd like to say I’m the first 
screenwriter to make this journey to the edit suite. But no, it turns out David Mamet has been 
here before. 


"Let the cut tell the story... You always want to tell the story in cuts. Because otherwise you 
have not got dramatic action, you have narration." David Mamet [5] 
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And I am also interested in the tools that film editors use. It seems to me the editor's 
interface bristles with possibilities for writers. The film is constructed in a timeline; each 
modification to the cut can be instantly replayed on screen. And in this way, moment-by- 
moment, the editor and director are testing the cinema-worthiness of their film. This is rapid 


prototyping. 
“Ever tried. Ever failed. No matter. Try again. Fail again. Fail better” Samuel Beckett [6] 


Both screenwriting and editing are essentially iterative and mending crafts. Improvement 
comes through failure. Editors can ‘fail’ quite rapidly, and thus arrive at a good cut fairly 
quickly. By contrast the tools and processes that screenwriters use tend towards slow failure. 
We're effectively still using a typewriter. 


If we look at other arts and crafts, we see most use prototyping as a means of perfecting 
design. A music composer can rehearse a phrase of music on the piano before writing the 
orchestral score. The sculptor has the maquette. The architect has scale-models, and 3d 
animations. 


How do screenwriters prototype their stories? They have index cards; and they have ‘Blake 
Snyder's Board’. And sometimes they have rehearsed readings. But all are locked in the 
medium of words. 


But it was not always thus. In the silent era, the image was sovereign. Here are two extracts 
from self-help books for writers from the 1920's. 


"We are studying photodrama. We must, therefore, learn to put our dramatic action into 
picture form... Our schools are beginning to give definite training in visualization..." 
Frederick Palmer [7] 


"Continuity writing... is a matter of improvising pictures which will render (these) ideas 
visible. It must make mental processes graphic.” Frances Taylor Patterson [8] 


And here's Eisenstein, very much outside the Hollywood system, in the heat of writing Ivan 
the Terrible in the early1940's. 


"When you are in a good working mood, images swarm through your busy imagination. 
Keeping up with them and catching them is very much like grappling with a run of herring. 
... you must drop your pencil and take up your pen to sketch the dialogue for this scene, and 
before the ink of this is dry, your pencil is once more making a note of an image that came to 
you during the dialogue... Whole scenes first take shape as batches of drawings before they 
take on the clothing of words". Sergei Eisenstein [9] 

During the silent era and into the talkies, the Continuity Script was the industry standard. It 
was a format that often contained a galaxy of visual, aural and technical details about camera 
angles, mise-en-scene and so forth. This script format gave writer license to prefigure the 
shoot and, to a high degree, the edit too. 


So what changed? How did screenwriters become detached from sound and picture? Here's 
a couple of key factors: 
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- The advent of the talkies in the late 20's tended to shift the screenwriter's focus away 
from images towards dialogue, and more ‘naturalistic’ ways of storytelling. 


- In 1940's, and for reasons we don’t have time to explore here, the Continuity Script 
was replaced by the Master Scene script - the one we use today. This format entirely 
stripped away the writer's license to include production descriptors, and references to 
images. 


"Writers are often discouraged from using images... It is only after the screenplay has been 
fully scripted that images come more fully into play in the form of storyboards, design 
sketches and other means of pre-visualisation... Implicit in the industrial model is the 
separation of words and pictures” Kathryn Millard [10] 


Kathryn Millard makes a seminal contribution to a small but growing body of literature 
around digital screenwriting. She wrote an essay called The Screenplay as Prototype, a theme 
she later expanded into a book called Screenwriting in the Digital Era. 


"In our digital world, the boundaries between the once discrete stages of writing, 
preproduction, production and postproduction are evermore blurred. Digital tools make it 
easier to cut, shift, morph and reassemble materials throughout the development and 
production process. The very notion of what it means to write is shifting. " 

Kathryn Millard [11] 


Previsualisation is one of the tools of our digital world. Previs usually means wireframe 
animations generated in software such as Maya. Their purpose is to allow directors to 
rehearse complex scenes like the car chase from Matrix Reloaded ahead of the shoot. 


This kind of previs, sometimes called animatics, is very expensive and generally undertaken 
by VFX artists and animators. 


But away from big budget action films, there is a range of cheap or free previs tools designed 
for screenwriters to use. Some of these work like storyboards, others like animations 
(Amazon Studio, Storyboard That, or Plotagon) 


As a screenwriter, I found these tools insular and restrictive; they tend to use a common stock 
of characters and settings. Most of them presupposed an already formed story. I was looking 
for something more open, more dynamic. 


My students are the generation ‘formerly known as the audience”, to borrow Jay Rosen's line 
[12]. They're digital producers, they make and mashup stories for youtube, vimeo, pinterest 
and the like. For this genereation: 


“...writing with text is just one way to write, and not even the most interesting way to write. 
The more interesting ways are increasingly to use images and sound and video to express 


ideas" Lawrence Lessig [13] 


I was looking for a hybrid tool that was both film editor, and word processor. I haven't yet 
found it. So I developed my treatment using two discreet packages: Scrivener and Prezi. 


My script is called Rush the Sky. I use the agility of Scrivener to compose story ideas, and in 
Prezi I convert these ideas into story beats, with proxy images and film clips from the web 
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(google images, flikr, youtube). These story beats and images are mapped onto this three-act 
grid. 


"Good ideas... are works of bricolage; they're built out of that detritus. We take the ideas 
we ve inherited or that we've stumbled across, and we jigger them together into some new 
shape." Johnson: Where Good Ideas Come From 


Though I’m using digital tools, this principle has a long history. At a recent Matisse 
exhibition the catalogue describes how his Cut-Outs were originally a way of prototyping his 
paintings: he'd make paper versions of the objects in his canvass, so that he could play with 
different compositions: let's put the Tahitian shell here, the apple over there and so on. 


And so with Prezi; I can move my beats and media fragments around, play and re-play them. 
But the really dynamic thing for me is this: in reaching for images, clips, and sounds, you 
enter worlds you could not have imagined. You have no idea what your search terms will 
turn up. This is what Steven Johnson, referencing back to Stuart Kaufman, calls the *adjacent 
possible”. The web is a place of extraordinary bounty, and serendipity. Here, the interplay 
between fragments of text, clips and images can be truly generative. Indeed, there is a strong 
connection here with soviet montage: the generation of new meanings through the collision 
of media. 


As I look at my story here, I can clearly see an A and B plot (this is a non-linear story, and 
these sections in blue are flashbacks). I can see a three-act structure, and I can see the entire 
story as a kind of diagram. And in a sense, I can play it. Though quite crude, by simply 
clicking through the beats of my story I have a flavour of the kinetic and temporal nature of 
cinema. Most importantly, like editing itself, this simple mechanism is a lie detector that 
helps make ‘plot holes’ visible. 


The treatment is where you build the bones of your story. This is the phase where the story 
should remain in fragments, and mobile. Software of this sort allows you to dance the bones 
‘til you have the steps entirely worked out. 


The next phase is the screenplay. The screenplay tends to fix the shape of the story and 
returns us to the medium of words. As a linear pre-formatted document, this stage is much 
less conducive to narrative experiment. 


One might argue that when we write our screenplay, we’re throwing out all the cinematic fun 
we had in the treatment. But actually, this is what will invest our writing; we are capturing in 
words all the vivid worlds, events, and characters we have seen with our own eyes. And the 

story will be strong, because in some measure, we have proved it for cinema. 


Pd like to give Kathryn Millard the final word: 
“The processes of screenwriting and filmmaking have been separated since the early years of 


cinema... Over 90 years later, the digital era offers the possibility of reuniting screenplay 
and film production in an expanded notion of the screenplay” Kathryn Millard [15] 
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The goal of our research is to enhance the traditional museum viewer experience by constructing 
an interactive exhibition piece where viewers can emotionally interact with the act of creating a 
fine art painting. Through our interactive Creative Artificial Intelligence System (CAIS) which 
generates art portraiture, we try to recognize what our users are experiencing (emotionally) when 
they view art work by considering their emotional responses in creating and evolving their 
portrait painting and using that as input within our system. Ultimately we hope that our CAIS 
system can bring unique learning tools to art education and appreciation by focusing on affect as 
one of the main elements within the interaction -- where emotion is constructed, interpreted and 
given meaning in a dialogue between the user and the generated art of the system, or between 
many art viewing users through a system. 


INTRODUCTION 


This research specifically addresses how our Artificial Intelligence (AI) tools not only facilitate 
and enhance the understanding of a creative process of an artist but also adopt aspects of affect 
and emotions that are aroused from viewing the artwork. Our research aims to grow the 
knowledge and cognitive model of our CAIS system that currently has aesthetic reasoning 
based on artist's creative processes to explore/understand the mood and emotion that 
artist/users' intent to portray in a produced artwork. This encourages new collaborative practice 
that allows for users to create a dialogue between the system and the users that can organically, 
share and exchange real-time knowledge, evolve their creative process and to guide their 
artwork through an emotional space. 


Our authored CAIS, explores the generation of new artwork using genetic algorithms where 
constraints are expressed as principles, rules and concepts related to visual organization of art 
and design compositions. The created output is a multilayered artifact based on a 
developmental processes defined by visual elements such as shape, size, color palette, texture 
and style. This was investigated in our previous work where we explored elements of the 
Futurist art master Umberto Boccioni's paintings are analyzed and input into the our CAIS, 
and a viewer/interactor can create many evolved new versions to explore/understand the 
underpinnings of the work through an experiential creation process [5,6]. This brings to focus 
the role of artists and designer as actor/agency working within cross flows of knowledge, 
perception and information. Hopefully allowing them to be a simultaneous participant and 
observer in ongoing collaborative conversation. We have begun to deconstruct the artwork as 
segmented rules, shapes and styles within our ongoing CAIS as an organism that can grow and 
evolve within each state of its development. This allows for a unique learning tool where 
museum viewers can use our interactive portraiture system to experientially explore the 
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creation of body of work through investigating texture, palette and brush strokes in general and 
through various user emotions. 


ACTIVE PARTICIPATION IN LEARNING 


Active participation is one of the most fundamental principles of learning and education 
theory. According to Dewey we learn best by doing, and it is simply insufficient to provide 
viewers with merely materials to look at [2]. European Union also published a LifeLong 
Museum Learning: European Handbook, that discusses constructivist theory as the desired 
model for museum education programs and encouraging the involvement of visitors within 
various activities, such as, making artworks and objects, theatre, role-playing as well as 
performance based activities [8]. It is also evident that such museum programs and activities 
are mainly geared towards kids, young adults and students rather than the general public. 
Learning experiences within museums are often built on wide range of materials, ideas and 
activities that surround the artwork rather than the materiality of the particular works [1]. What 
is significant and valuable to learning are the connections made by the audience and not just 
their direct exposure to the artwork alone. We also understand that visitors learning preferences 
and motivation for learning may vary, so we wanted to add to the museum environment a 
multimodal experience that is self directed, customizable and caters to various learning styles 
simultaneously, a critical factor for effective learning. 


Therefore we wanted to design an interactive art education experience that focuses primarily 
on this notion of active participation in the museum environment. Not only do we want our 
interactive experience to act as a supporting material but also to supply viewers with wide a 
range of experiences. Our affective CAIS system invites visitors to participate in activities 
related to the exhibition theme, say Futurism or the color palette and stroking style of late Van 
Gogh, (Figure 4) encouraging them to engage their imaginations in response to the ideas 
provoked in the exhibition through exploration of artist's creative process and the act of making 
an artwork in a specific style and mood. This add-on interactivity acts as a catalyst for learning 
by providing new experiences and creative challenges; enticing audiences to further engage in 
the materials/ideas presented as part of what the museum has to offer. It acts as an analyst and 
bridge to fully involve viewers with the presence of the artwork and the artist's intent in the art 
museum setting by allowing for a participatory dialogue. Through our system, we want to 
encourage the viewers to explore a painting in depth and experiment by undertaking a personal 
journey and substantiate their personal narratives through emotional navigation in response to a 
selected artwork or art series. 


Incorporating affect and affinities within interactive experiences can be challenging as they 
need to be open-ended in order for the users to be able to orchestrate their own emotional 
experiences. Hence, we wanted to provide the visitors with an activity to construct their 
experience out of what is important and meaningful to them, allowing for that personal 
encounter. To not only capture their attention but to engage them through a tool that stimulates 
and improve that inquiry cycle and sustains that further, both cognitively and emotionally. 


INTERACTIVE EXPERIENCE 


We use Artificial Intelligence (AI) tools that take as source a photo in front of our digital 
canvas and through their selection of an emotion (how they feel) create and explore a new 
generated artwork (Figure 1). This photograph as source is evolved into a final art portrait 
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work that uses the rules, styles and elements from our CAIS portrait painting system (as well 
as our labs on going research in historical art styles) and maps different texture, brush stroke 
styles and palette to the various emotion based on survey studies. This allows for an art 
education component to the work as we can use styles, palette and other factors in our CAIS 
system from a known traditional piece, an artist's body of work, or art general as we have 
attempted with Rembrandt [3], Picasso [4] and Boccioni [5] in past research work. In this way, 
we do not want to reduce the viewer's experience into a few variables or simple templates of 
styles. But instead use the more open and human sense of emotion as the core of what humans 
are and how we experience the world. This is an open research area, so understanding the right 
mix of templatizing for non-artists to be able to create something of worth versus a full 
interactive control for experiencing the process in a personal way requires several iterations. 
AII aspects of the final images from our interactive are created generatively with texture 
synthesis, cognitive synthesis (using the palette and rules of portrait painting algorithms) as 
well as results validated from our emotional mapping survey studies. 


Figure 1: A museum viewer approaches our system, has their portrait taken, then via emotional choices is able to 
explore the knowledge/experience space of fine art creation -- creating artwork through presence and emotion. 


The original artwork or deconstructed seed “gene” objects are fed into the system [3,4] are not 
a template or design schema, rather pieces of knowledge (genes in evolutionary terms), content 
that can be fundamentally evolved through means of an interactive control. This creation and 
exploration process through the rules, styles and visual objects of an artwork gives the 
viewer/interactor an opportunity to explore the creation process of the artwork. Through our 
CAIS portrait system, the viewer/interactor can exchange, manipulate and evolve the 
deconstructed seed forms, their photograph based on emerging ways an artist's might negotiate 
aesthetic elements within their work (Figure 2). By the selection of different emotions, our goal 
is those interacting with our CAIS portrait systems can select an emotional space through 
transfer of knowledge from the provided domain. Our CAIS environments simulates the more 
deeply human cognitive process by attempting to depict a still work, not from the visual 
elements segmented out of the historical canvas, but also from new source, a photo portrait of a 
viewer in front of the digital canvas. 
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| am feeling 


PE SLIK TH en 


Figure 2. This is a depiction of the system in process, a photo of the viewer is taken, emotion choices are made by 
the viewer, processing occurs and one to several portrait styles are made based on emotional input. In future 
interactions, we are investigating using the user’s mobile device as well as bio-sensing emotion. 


We have conducted a study to understand the mood, affect and emotion that are evoked from 
the artworks produced by our authored CAIS. Our study has provided us with preliminary data 
to inform and define various validated emotional spaces within our CAIS environment 
according to the results from our user study surveys. The surveys identified corresponding 
emotions selected by majority of viewers of the various artwork (recipes) produced by our 
CAIS. We concentrated mainly on mapping of the mood to texture, brush strokes and palette. 
We used the emotional mapping system, based on Russell's circumplex model of affect [8,9, 
10]. This model encompasses four quadrants, each representing a set of emotional category 
based on positive and negative levels of arousal and valence. For the purpose of our study, we 
chose the following 12 emotions (3 from each quadrant of Russell's model): excited, delighted, 
happy - satisfied, relaxed, calm - tired, bored, sad - frustrated, angry and afraid [8,9]. Our goal 
was to match visual attributes to certain emotions that were stimulated in our participants by 
the artworks produced from our CAIS regardless of the content and subject matter. 


CAIS UX 


The shift from a research based to practical application of CAIS systems such in museums 
requires attention to the collaboration and creative processes of the users (artists/viewers). The 
design of such systems needs to not to only support their cognitive load of the process, but 
provide users with an engaging environment where their interaction with the system becomes 
transparent and provides a poetic experience that resonates immediately but allows for 
reflection and continues to inform later. It must have an essence; where it is meaningful yet not 
trying too hard — it needs to be intuitive. There is a fine balance between conveying art 
education, but yet providing the user with an exploration tool. Part of our research especially in 
the UX domain is to allow for both knowledge gathering (art education) and knowledge 
exploration. This shift of focus from knowledge taxonomies to problem-solving situations 
provides a more engaging educational experience to visitors. 


87 


It is important to value the role of the human collaborator and user experience while working 
with computational creative systems and to create a dialogue between the system and the users 
that can organically, share and transfer real-time knowledge and evolve their creative process. 
Some generative and AI systems value the final result over user input — meaning the interactive 
input of the users only minimally affects the final results. This problem is seen allot in early 
interactive systems. Our goal is to keep a high final output quality but assure the user 
participation is paramount to the process — without it, the reflection and personal experience 
aspects of education are diminished. In this way, the integration of user experience within 
computational creativity systems encourages new modes of creative practice where users 
remain fully engaged in their creative process and exercise their creativity through experiential 
learning, reflecting and creating. Such engagement can be explored through various modes of 
sensory input, new affordances, experiences, and interactions with the inclusion of 
embodiment and movement taking into consideration interactivity, engagement, collaboration 
& communication. 


Our CAIS portrait system explores new modes of engagement with art in traditional museums 
through various user experience (UX) strategies based on our studies. It is important goal to 
create harmony between the modalities of their interactive experience, from the initial 
encounter with the system to the final generated artwork they create (and explore). The system 
attempts to use user emotion as both a social embodied process in art and painting, as well as a 
notion of a traditional artist's expression and intent. We want to focus on the emotional journey 
while interacting with a generative art piece and involving our users/viewer in the meaning 
making process. This spiral of experiencing, reflecting and making provides a participatory 
platform for co-creation of ideas, which is visitor focused, open-ended and prepares the viewer 
for broader, richer, learning experiences. This user experience process, by exploring the 
various palette, brushstrokes and texture which can add to the traditional art viewing process 
by providing the audience with an interactive experience to better understand and connect with 
the traditional painter's possible authoring experience when they paint, or at least to personally 
experience the palette, brush stroke style, and other aesthetic choices of that artist (Figure 3, 4, 
5). This experience of interacting with art creation allows the viewer to connect to a traditional 
painter's creative process and their expressions through a shared meaning, allowing for a 
deeper personal appreciation of their work. 


Figure 3. Example of an exploration of palette and stroking style of the Impressionist era of Claude Monet's 
"Sunset in Venice" 1908 using our CAIS personal portrait from our system (right). 
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Figure 4. Example of an exploration of palette and stroking style of the Post- tion era of Vincent Van 
Van Gogh 1889 using our CAIS a personal portrait from our system. 


Figure 5. Example of an exploration of palette of the Post-Impressionism era of Vincent Van Gogh 1889 using 
our CAIS personal portrait system. 


Our system also offers different levels of possible engagement as viewers can choose to step 
back and observe the interaction of others with the system or go through the archive of past 
evolved portraits. This allows for stimulation and continuity of experience through engagement 
with the past archived portraits that users have made, each tell a different but related story. 
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They are also able to share their creations (digital portraits) with friends (Figure 6), which 
promotes the social component and shareability both in the museum and in a large social venue 
that can support trips to the museum. Working with the museum, there is a social media/viral 
marketing aspect where once setup, users can email a URL of their portrait to others to share 
their experience, both branding the exhibition and inviting a larger community to come 
experience the work by visiting the museum. This experience also allows for personal 
storytelling, where the viewer can formulate and develop their own narrative of their portrait 
through the selection of various emotions. Identity, like interest, develops through interaction; 
both interest and identity develop in relation to available experiences and to how learners 
perceive, understand and represent these experiences. Our interactive process provides an 
environment where viewers become co-authors; through participating in the creation of the 
portrait, and personalization of the artwork based on their mood and emotion that only relates 
to them in that moment of interaction. This interaction enables viewers to learn by creating 
knowledge through experimentation and formation of purposes driven by their curiosity, 
interests and emotion, rather than simply transferring it. 


Figure 6. Source photo (left) with examples of unretouched, purely software generated output via our emotion AI engine. 


CONCLUSION 


This research aimed to enhance engagement and active learning in museums by facilitating the 
audience's understanding of how emotion and mood can effect the traditional artists creative 
process and their art through their own exploration of various affect while interacting with our 
generative painting system. Interactive and experiential learning in museums is beneficial for 
both the participant (the learner) and the experience provider (the museum); as the visitors 
develop an emotional engagement and continue to share their reflections. Using specific CAIS 
based generative digital tools and hardware setup, this experience can enrich their personal 
connection to the traditional artwork through the appreciation of not only its physicality; 
palette, brush stroke and texture but also its ecology, time, history, intricacies and its mood 
through an emotional navigation. The viewers can now value and immerse in the artists" 
creative process and weave a personal narrative based on their experimentation. Care and 
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iterative design and refining cycles are needed to be both respect the traditional art and be able 
to explore / reflect on some aspects of that art via another (1.e. the viewers generated portrait) 
interactive medium. For future, work we want to refine the system in real world sceneries and 
we want to dig deeper into full body experiences, since emotion is a social and embodied 
process and there are multiple ways of expressing, sensing and feeling emotion such through 
movement. 
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Abstract 

Cultural Heritage may represent the appropriate instrument in order to reach specific global problems, it 
can generate scientific excellence and cooperation between third countries; in a cross-border and 
intersectoral perspective. It can contribute through the use of innovative tools, to the enhancement of the 
artwork through the research of “truth” and the perspective of a “planetary humanism”. 

The restoration of an unpublished picture: the preservation of the material. 
Through the identification of a scientific method and the use of advanced technologies, it is possible to track 
and correlate all the information related to a product and how this strategy can lead to the retrieve of the 
truth and the enhancement of the artwork. 

An example of the method on an unpublished painting. 


Introduction 


Every "single form" of cultural expression has to represent a key role throughout the world's 
Cultural Heritage. 

The characteristics of this sector highlight the vastness and diversity of the artifacts. Such 
peculiarities, while causing considerable obstacles to the solution of a number of “criticality” (as 
cataloguing, safety, traceability, research, etc.), are the essential link between different cultures and 
also the tool for their “coding”. 

The understanding of the bonding between the processes, the interactions, the implications and 
also the conflicts and uncertainties, represent an innovative synergy in order to cause a complex 
scenario of generative diversity; oriented in a "planetary humanism" perspective. 

The more this bonding comes understandable and usable, the more we will have fulfilled one of 
the fundamental concepts expressed by sociology: the establishment of the development conditions 
through new acquisitions of knowledge. 

SMARTICON arises from the need to provide answers to those needs. 

*The noises of the world, weapons, conflict strike me deeply. The radical nature of the crisis of 
society and humanity prompted me to search for the root level of the theory, towards a progressive 
reorganization of the structure of knowledge" [1]. 


Methodologies for restoration and preservation of Cultural Heritage 


We are going to examine a painting from the late Sixteenth and early Seventeenth century. 

The work, executed with oil on canvas technique, depicts a “Madonna with Child and small 
dove". The artifact is in excellent condition. 

The restoration has been followed by myself. 

A preliminary survey has detected the presence of a previous restoration, in the course of which 
the fixed frame has been replaced with a movable frame and ancient canvas has been submitted to 
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sheath. Today the pictorial material shows a painting in which transpires some whitish “blooms”, 
due to a traumatic straightening performed during the old sheath. 

Nowadays technical intervention has opted for the preservation of the old sheath because, being 
capable of fulfilling the function of supporting the original material, its removal could have caused 
further trauma. 

A thorough cleaning allowed to remove the old paint and touch-ups performed during the 
previous restoration. Nowadays pictorial reintegration is performed “by body" (invisible 
reintegration), in order to restore only some limited localized gaps, mainly, in the extreme lower 
end of the painting. 

The work shows the extraordinary quality of the execution, also reflected by the very expensive 
colors of the palette of pigments used. 

In different parts of the painting are detected obvious "second thoughts" and 
traces of date and signature. 

On the lower right a label is glued which probably refers to a number of inventory. The numbers 
can be interpreted as “370”. 


The diagnostic investigation 


The painting has been put through reflectography pushed to 1700 nanometers (IR-1700) and 
radiography X Digital (DRX), in order to investigate which technique was used and the conditions 
of preservation. 

Infrared 1700 guarantees to obtain more detailed feedback compared to a normal IR 1100, since 
the pigments, at strong wavelengths, become more transparent. 

This technique allows a clearer vision of the underlying layers and ensures a more reliable 
feedback of the pictorial drafts, regrets, preparatory drawing, signature, the identification of 
retouching, repainting, gaps and abrasions. 

The excellent conditions of the painting have been found in the small percentage of plastering 
and reintegration, mainly affecting the extreme lower part of the painting. 

The archaeometric appearance shows many "regrets" particularly in the compositional genesis of 
the work. 

The outline of the face of the Child, in particular the chin and cheeks, was originally positioned 
lower compared to the drafting of the final color. This variation is also evident by looking at the 
painting with a naked eye and noticing, how the final color drafting lets a glimpse, in transparency, 
of the first version of the painting. 

In a first draft next to the face of the Child, tree branches originally invaded the bottom of the 
painting almost touching the left shoulder of the Child. In the final draft, however, more space is 
given to the figure which stands out from the background on which the blue lapis lazuli covers the 
previously painted leaves clearing the figure of Jesus. 

The right arm of the Child has been moved lower compared to the first draft, and both hands 
have been repainted many times in order to find the final position. On the hands of the virgin you 
can notice different movements of the fingers and some changes on the folds of the blue drapery: 
next to the pink sleeve and on the fold that rests on table next to the pages of the open book. 

To follow are compared images of the painting and the images from the IR (Fig. 1 and Fig. 2). 

The second thoughts described earlier are better shown in the images below, highlighted 
throughout the comparison with the IR. This image clearly shows the outlines between the first 
drafts and the final movements (Fig. 3 and Fig. 4). 
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Figure 1 Figure 2 


Figure 3 (IR image) 


Figure 4 (IR image with overlay gloss) 


IR image with overlay gloss (the darkest lines delineate the final draft, while the thinner ones 
identify the first draft, amended during construction) 


The transparency of some additional backgrounds of lapis lazuli, although it may lead one to 
evaluate a possible afterthought in the distribution of light and shade of the mantle of the Virgin, 
testifies instead a refined technique; brilliantly designed by the artist to push himself to achieve an 
exaggerated contrast called and suggestive transparencies, which could not otherwise be achieved if 
not by layering transparent glazes. 

The artist has also accomplished the extraordinary blue drapery without parsimony using pure 
lapis lazuli (as confirmed by chemical analysis of the pigment). 

Through this generous color writing, the entire surface of the Madonna background light up in 
order to apply the overlay gloss which once dried show the latest dark glazes: appointed to shape 
the shadows, but intended to let perceive everywhere the underlying layer below the intense luster. 

It should be noted a regret in the preparatory drawing: on the high profile and along the book. 
The drafting color for the making of the cover reduces thickness and volume, hiding parts of the 
drawing which produced more pages now no longer visible. 

The painting technique highlights the presence of “edges in savings", or colorless material 
(colored backgrounds that do not reach the limit of volumes, evidently to follow the underlying 
signs in order to obtain a three-dimensional affect) and the author's tendency to mark the details 
with brushstrokes of a brown outline, above the paint layers. 

This technique, only used by great masters (as it will be for Caravaggio), grants to have a fast 
color spreading which allows to avoid that the fresh colors do not get mixed with the adjoining 
ones, so that it is not necessary to wait for the color to dry on borderline of other pigments. This 
technique also allows to obtain a better volumetric rendering, since the figures are detached in a 
precised way from down below. 
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Finally a micro sampling was performed under the right elbow of the mantle of the Madonna, 
which confirmed the characterization of the blue (Fig. 5 and Fig. 6). 


Figure 5 


Figure 6 


On the bottom left, next to the lectern, drawn with a medium point brush, some letters turn up as 
wellas three numbers on the bookbinding. 


Figure 7 


The numbers painted on the book spine indicate: “593”. It is 
reasonable to assume that they can be identified dating: "1593". 

On the lectern there are two initials "A" and "C" (the painter's 
initials) and, in the center and slightly below, a "P" (pinxit) (Fig. 7). 

Both the dating and initials, can be traced back to the works 
autographed by Annibale Carracci who, in those years, was working 
on the decoration of the Palazzo Farnese in Rome. 

Another crucial element, which places the painting under the 
prestigious Roman patronage, it is an engrave by P. Daret , which 
suggests that the original painting was made by Annibale Carracci 
and came from a Roman collection. 

Below, to prove the authenticity of the acronyms, shooting infrared 
light into visible light with 
high contrast positive (Fig. 8) and negative (Fig. 9). 
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Ma A 
Figure 8 Figure 9 

Digital x-rays X (DRX), highlight the original textile support , with armour canvas and irregular 
and thick spun yarn, ascribable to an handcrafted. 


The search for “truth”: SMARTICON 


SMARTICON is a scientific innovative instrument, able to manage all the "complexity" of the 
Cultural Heritage sector, which identify the "criticality" and "limitations of the current technique". 

The method of placing an hermeneutic order between the different information that 
populate the art field. 

Through repetitive and methodical processes , it allows the information to generate 
reports and correlations and, above all, to bring out contradictions and to manage them so 
that they become enunciated: that turns current "critical" in the aspired product, currently 
foreclosed, which generates the regain of knowledge and the pursuit of truth. 

This tool is in charge of the organization, to the regain and usability of all those information that 

represent a priceless wealth of knowledge, but it is currently foreclosed due to the vastness and 
diversity of information. 
The method consists of a collection of empirical evidence through the observation of experimental 
information and strategies (up to the formulation of statements and rules in order that they become 
"valid"); but also in the formulation of hypotheses and theories that represent solutions for the 
continuous implementation of the processes. 


This tool is in charge to put in order all information, to track, to highlight and address the 
contradictions, discontinuity, the critical (always there in the cultural Heritage field ) and all the 
interrelationships referable to an artifact ; finalizing the process of recomposing, to the integration 
of all information, through a complex and profound analysis: the identification of objects 
portrayed, tracing data, historical and geographical localization(and much more). 
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Let's go back to the painting 


SMARTICON is a method through which all information that populate the art field, take up an 
hermeneutic order and are transformed into sentences. 

The advantage is that all statements, by their nature, have the ability to generate relations and 
correlations between them, making all information comparable and useful. 

The method also provides a complex system of “functions”, designed to establish the parameters 
and search tools which allow to define the "rules" for the good functioning of the method in order to 
retrieve automatically all information of the work of interest. 

SMARTICON makes it possible to generate the regain of the knowledge that leads to the 
"enhancement" of the artwork. 

Let us now examine all the information arising from the examination between the relation and 
correlations relevant to the artwork in discussion. 

The method underlines there is an ancient text with the description of the work discussed. 

The painting depicts a "Madonna and Child". The description corresponds to the one shown in 
Malvasia (Part II page 88): “Annibale tagliate” “Una Madonna, mezza figura in faccia, che 
appoggia ad un tavolino, guarda su un libro aperto; sostenendo con ambe le mani incrocicchiate il 
Signorino in camicia, in piedi, che attenendosi con la sinistra al manto, con la destra fa volare la 
rondinella appesa al filo". 


Figure 10 Figure 11 Figure 12 


The SMARTICON process as been able to identify some engravings and drawings, that can be 
identified in the work discussed. 

Observing the engravings and placing (Fig. 10 and Fig. 12) them in comparison with the 
painting, it evidently appears that they are enriched with numerous decorative details more than 
the current version that, probably for a rapid execution and freedom of touches, is the first version 
from which can derive a second more defined version. 

This work refers to simplicity, essentiality and the *immediacy", unlike any subsequent versions 
more processed and totally finished. 

There isa very interesting comparison with the beautiful drawing by J. Reynolds (Fig. 11) that, 
in its essence, refers precisely to the work in discussion. 
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A key step: method validation 


A key role is the experts proof, who will evaluate, approve, deny or integrate 
results. 


Acknowledgements 


A very special thank you goes to Prof. Mina Gregori, authoritative art expert especially on the 
Emilian paintings of the Sixteenth and Seventeenth centuries, who assessed the work by writing the 
following text: 

"This intense work is a significant addition to the catalogue of Annibale Carracci, and I am sure 
will be welcomed by scholars. 

By looking at the format, it was clearly committed by private clients, the canvas is in an 
excellent state of preservation. The looseness of the brush strokes and the perfect color combination 
offers us an image of intimate gentleness and humanity. The Child and the Virgin, are ivolved in a 
tender embrace, while the Virgin is reading and the Child is trying to make a small dove tied to a 
wire fly. The hand of the Child clings to the cloack of the Mother with intensity, as if to prevent a 
future separation from her. 

The work is signed in monogram by the artist, corresponding to one of those known to Annibale. 
Reflectographic investigations have highlighted rapid drafting brushstrokes and many regrets 
during construction. Among these the most evident is the presence on the right of a tree branch that 
in the final execution was covered from heaven. 

Precise findings are noted with other works by the artist, as the 'Madonna Montalto' at the 
National Gallery in London and the 'Pieta' of the Kunsthistorisches Museum in Vienna. 

I intend to deepen the study of this work in order to write more details in my forthcoming 
publication. Prof. Mina Gregori" 


Conclusions 


The method gives life to a flexible and dynamic digitization of the artwork and allows you to 
arrive at the level of knowledge currently foreclosed, generating benefits in cataloging, security, 
traceability of the artwork and all the information related to it, enhancement, promotion, 
incrementing tourism, education, research, cultural exchanges, aims to strengthen the base of 
science and technology, to promote research, to address the specific global problems (the 
smuggling of artworks and heritage), to generate scientific excellence and a cooperation 
between the European Union and third countries and to facilitate a dialogue between different 
cultures. 

The characteristics of this project allow to apply this method to any type of manufactured article 
and also to Eastern culture heritage, taking place in an important strategic International scale. 
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Abstract — With roughly 2 million images and the BildarchivOnline image database boasting 
around 300,000 images online, ETH-Bibliothek’s Image Archive is one of the largest and most 
present image archives in Switzerland. However, very little is known about the behaviour of the 
Image Archive’s users. This paper discusses the following issues: has its usage changed in the 
wake of the increasing digitisation of the images? Which holdings are used the most? Is there 
development potential for individual holdings? Who are the people that use the Image Archive 
and are there differences between these user groups? Which individual images are in demand 
how often? Which factors influence the use of images? How would a possible change in the 
licensing model in favour of open data influence usage? In order to answer these questions, the 
entire usage of the Image Archive from 2001 onwards was analysed. This involved evaluating all 
roughly 3,000 usage agreements with a total of 14,506 ordered images quantitatively. 


INTRODUCTION 


ETH-Bibliothek’s Image Archive was launched as an organisational unit in 2001. With 
roughly 2 million physical images dating from 1860 onwards and the online image database 
BildarchivOnline, which contains around 300,000 images on the internet, it is one of the 
largest and most present image archives in Switzerland. It has been digitising selected image 
holdings since 2001 and presenting high-resolution images in its own online database 
BildarchivOnline since 2006. Low-resolution images have been available for free download 
under the Creative Commons Licence BY-NC-ND in Google Image Search since the 
beginning 2009 and on BildarchivOnline since early 2014. The annual output of new images 
online has risen sharply in recent years, amounting to around 40,000 images in 2013. 

One issue in image science is the extent to which the switch from analogue to digital 
images has changed the use of images. So far, there have been very few studies on the 
behaviour of the users of scientific image archives. This paper discusses six questions on 
usage based on the user figures since ETH-Bibliothek’s Image Archive was launched. 


Method and Overview of the Holdings 


Two data samples were available to analyse the usage: in Sample 1, all the usage 
agreements for high-resolution images since the creation of the Image Archive were 
evaluated, where the purpose of use and all the individual images are documented with call 
numbers. These 3,000 usage agreements with a total of 14,506 images ordered from 2001 to 
2014 were transferred to an Excel list. Moreover, the images have been available for free 
download in web quality directly via the image database since the beginning of 2014. This 
new form of usage forms the basis for Data Sample 2. 27,952 low-resolution images were 
downloaded between January and the end of September 2014. The downloads were analysed 
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according to image call numbers. Furthermore, the access figures for the image database 
BildarchivOnline were included in the analysis. 

For the interpretation of the results, the holdings were divided into seven sub-holdings: 
View Collection; Portrait Collection, specialising in ETH-Zurich professors; Photographs in 
personal papers, specialising in ETH-Zurich professors, people close to ETH Zurich and 
associations; the photographic agency Comet Photo AG; the Adolf Feller Postcard Collection; 
the Swissair Photo Archive; and Aerial Photographs from Comet Photo AG and Swissair. 


Results 


Six questions on usage are discussed here based on the analyses. 

Question 1: Has the usage of the Image Archive changed through the increasing 
digitisation of the images? The number of orders first began to increase significantly in 2009. 
This can be attributed to the larger number of images that have been uploaded onto the 
internet every year since then, but also the greater visibility and findability of the images. 
Moreover, the images can be found in web resolution under the Creative Common License 
BY-NC-ND via the Google Image Search since February 2009 and via the integrated search 
on the library homepage since June 2010. The publication on Google also resulted in a highly 
significant increase in hits on the image database BildarchivOnline. On the whole, the Image 
Archive has become more visible through the mounting digitisation of its holdings and 
publication via various platforms. The usage between 2008 and 2013 increased by a 
considerable 560 95. 

Question 2: Which holdings are used the most? 

Figure 1 displays the usage according to holdings from 2010 to 2014 with a total of 9,584 
images sold. A total of 14,506 images were sold throughout the entire period examined. 
Although the demand for the Personal Papers holdings specific to ETH Zurich was very high 
on the whole, 32% can be traced back to one single scientific user in 2010, for instance. The 
same year, with 10% of the overall demand, the new Feller postcard holdings were popular. 
Whilst the Feller holdings were present in the media, they were also in demand considerably 
more frequently (2010 and 2011). The first Swissair images were also published in 2010. 
Moreover, the second volume of the Image Archive's book series Image Worlds [1] was also 
devoted to these holdings in 2012. The usage increased considerably during the publication 
year before levelling off again. The aerial photographs, however, have been a huge hit. The 
first aerial images from the Swissair holdings have been published since 2012. The usage 
figures rose from 24% in 2012 to 40% in 2014. The fourth volume of Image Worlds [2] was 
published on these holdings at the end of August 2014. 

Question 3: Is there any development potential for individual holdings? Three different 
datasets were compared to answer this question: the 2014 downloads and sales on the one 
hand and the sales throughout the entire period on the other. The aerial photographs harbour 
the most development potential: they were downloaded in web resolution and ordered in high 
resolution the most frequently in 2014. They have already climbed into second place behind 
personal papers in Sample 1’s overall use. In the case of downloads, the Feller and Swissair 
holdings were in high demand. It is a different story for the portraits and Comet press images, 
however: the sales in the same period reveal precisely the opposite behaviour for these 
holdings. Portraits and Comet press images were mostly used for publications in a printable 
resolution. There is great potential in the ETH-Zurich-specific holdings, which, besides the 
personal papers, also include the portraits and views, but also in the Comet press images, 
which are especially ordered frequently by commercial users. 

Question 4: Who are the people that use the Image Archive and are there differences 
between these customer groups? The users of ETH-Bibliothek's Image Archive are divided 
into three user groups: scientists and private and commercial users. The most important 
customers are the scientists (76%). Whenever possible, private users (13%) are referred to the 
lowest-resolution images on Google or, since 2014, downloads to reduce fee-based usage. 
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The most popular holdings according to user groups paint a differentiated picture of user 
behaviour (see Fig. 2). 

The three user groups order different holdings. Scientists use personal papers, views and 
aerial photographs; private users, on the other hand, tend towards aerial photographs, Comet 
press images and personal papers; and commercial users are especially interested in Comet 
press images, personal papers and portraits. While portraits also play a role in scientific 
publications, their private use is negligible. Overall, personal papers are ordered the most 
frequently, followed by aerial photographs and views. 

Question 5: Which individual images are in demand how often? Sample 1 was analysed 
according to images that had been sold multiple times. The best-selling image was the group 
portrait Three Members of the Olympia Academy featuring Albert Einstein (record name 
Hs 1457-71). The image was sold thirty-one times throughout the entire period examined. 
The five best-selling images were also Einstein portraits. It should be mentioned that Albert 
Einstein is ETH Zurich’s most famous graduate. Against the backdrop of the results in 
Questions 3 and 4, it is also interesting that seventeen of the thirty-two best-selling images 
were portraits, the majority of which were individual portraits and fewer group portraits. 
Users therefore specifically search for people related to ETH Zurich in the Image Archive. 
Furthermore, it is clearly evident from the individual purchases throughout the entire period 
that so-called icons and famous images were particularly popular: Max Frisch and Friedrich 
Dürrenmatt in the Kronenhalle in Zurich was ordered fourteen times (Com L12-0059-8018), 
the Eiffel Tower sketch by ETH-Zurich graduate Maurice Kóchlin seventeen times 
(Hs 1092), and the Greenland expeditions of ETH-Zurich Professor Alfred de Quervain 
fifteen times (Dia 297-0071) and eleven times (Dia 297-0073) respectively. Most users 
ordered familiar items, while very few set off in search of special or even unknown image 
material. This rubric also includes images that have already been published, such as pictures 
published in various publications to commemorate ETH-Zurich anniversaries or the cover 
picture of the Image Worlds volume on the Swissair holdings, which was sold ten times 
(LBS SR03-09918-11). The analysis of Sample 2 according to individual, low-resolution 
images that were downloaded several times, however, reveals a completely different 
behaviour pattern. Individual icons such as Einstein portraits or the Greenland picture 
(Dia 297-0071) were also frequently downloaded. Almost half of the multiple downloads, 
however, were aerial photographs. ETH-Zurich-specific images, such as the view of the main 
ETH-Zurich building, were also popular. Downloads are reserved for private and scientific 
use (see also the results in Fig. 2). 

Question 6: Which factors influence the use of images? Two key factors can be identified: 
on the one hand, the publication of the images via different publication platforms. The images 
can be used via the Image Archive's own image database BildarchivOnline since 2006, and 
have been displayed in Google Image Search since 2009 and in the integrated library 
catalogue on ETH-Bibliothek's homepage since mid-2010. The underlying strategy for the 
multiple publication of the images is to go wherever the user is, which especially applies to 
Google. Another effective instrument to increase usage figures are marketing measures, such 
as targeted press releases or other reports on individual image holdings or the publication 
series Image Worlds. The more users are reached, the more usage is generated in the short 
term. A report on the aerial photographs on the evening news programme Die Tagesschau on 
Swiss television in February 2013 resulted in the highest user numbers. 


Conclusions and Outlook 


The usage has increased significantly through the onset of digitisation, diverse marketing 
measures such as press releases or targeted book publications, and the publication of digital 
images on various additional online platforms such as Google Image Search or the library 
catalogue. The strategy of digitising selected and partial holdings is being pursued further. 
Aerial photographs, personal papers, views and portraits meet the users’ needs. Famous and 
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published images and icons are purchased as high-resolution files the most. As for 
downloadable images in web quality, the aerial photographs are in demand the most 
frequently. 

The images are offered to users as independent source and research material. In the age of 
big data and digital humanities, the mass of 300,000 online images is waiting to be searched 
through, analysed and visualised. An Image Archive's task here is primarily to digitise and 
index the data and make it available. The previous use of licensed images, however, reveals 
that users only utilise very few, mostly famous images for illustrative purposes in scientific or 
commercial publications or, at most, aerial photographs for evidence purposes. 

It will be interesting to observe whether the introduction of open data in ETH-Bibliothek's 
Image Archive scheduled for 2015 will raise image-science research questions where (large) 
image corpora are used as base and source material for scientific issues. So far, this kind of 
usage has taken place with the Image Archive's images in rarer cases or, in most usage cases, 
the holdings of the Image Archive have been included as individual pictures as supplementary 
material. 

In order to assess the users’ needs and the online user behaviour even more effectively, the 
search terms employed by the users would need to be analysed based on log files in 
BildarchivOnline in future. 
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Figure 1: Usage according to holding (2010-2014) (N=9,584) 
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Figure 2: Holdings according to user groups (2001—2014) (N=14,506) 
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ABSTRACT 


New technologies have opened new perspectives, ways and practices of telling stories 
about archaeological tools. For reliable identification of archaeological stone tools function 
analysts rely on multiple lines of evidence that might include tool design, use-wear, 
residues, breakage patterns, hafting traces and archaeological context. Ethnographic and 
experimental evidence is also important. Use-wear and residue analysis has the potential 
to provide a reliable basis for reconstructing and evaluating the nature of prehistoric tasks, 
resource utilization and settlement history. The use of these integrated, interdisciplinary 
sources, including documents and images from the past, makes it possible to identify the 
object's genesis. 

The paper will draw on case study from Upper Paleolitihic site of Bilancino (Tuscany) 
grinding stones to show how and with what degree of perceived success, digital 
technologies can be successfully employed in the process of dealing with archaeological 
data, materials and knowledge and how digital tools can be helpful in order to merge, to 
edit and finally disseminating integrated analytical approaches. 

The identification and description of use-wear traces was carried out by means of the 
innovative application to wear-traces analysis of the combined potential of the Digital 
Microscope (Hirox KH-7700). It has a 3D multi-viewer function that allows to observe 
surface shape at various angles. Two different optics are available: macro MX-G 5040Z 
working as a stereomicroscope at lower magnification and OL —140 Il working as a 
metallographic microscope (up to 7000 x). 

A fully-focused image can be obtained instantaneously by compiling images of different 
focus positions, building a 3D model that enables versatile observation in three dimensions. 
This device has proved a very effective tool for distinguishing between the warned crystals 
embedded in the matrix of the slab. 

A further step in the study of the grinding stones included their digital acquisition through a 
3D laser scanner. Two approaches were carried out: public dissemination and detailed 
research analysis. The dual approach led to the use of two different processing 
methodologies and visualization platforms (3D portable document format and tools for 
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multimedia interactive web). 

The use of open source solutions for 3D contents publication on the web was chosen 
because the modular system of open source software allows to customize tools and 
interfaces. This approach enhances the interactions with the virtual scene and the 
understanding of the 3D representation of the real object using annotations and selective 
visualization on the 3D model. 

The step forward in integrating cross-media approach to browse stone tools has been 
represented by the elaboration of a digital exhibit. The digital exhibit we are presenting can 
merge both the analytic data with the other kind of data in order to enable a different and 
multi-layered storytelling. 

Digital storytelling exploits the potential offered by digital tools for editing and 
disseminating contents, to create an enriched narrative process thanks to multimedia 
cultural resources. This is a major opportunity for cultural heritage to pass from digitization 
and aggregation of content to edit this content, and thus for cultural institutions to make 
their content more attractive for the public, providing new experiences and ways for 
scientists and for engaging with new audiences. 
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ABSTRACT 


We would like to show the potential of a holistically integrated culture cloud by the example 
of a tourist's journey to Florence. 

Following various stages of a visit to the town of Florence we demonstrate how digital user- 
expectations of a culturally interested person can be satisfied - all with the help of a single 
search-and navigation assistant: the E-Culture Cloud. 

Particularly we would like to address how a holistic use of user-data will benefit both 
institutions and visitors and serve moreover as sensible supplement to the individual's social 
web. 


THE CULTURE CLOCK 
The Model 


The stages of a culturally interested tourist can be systemized - since every visitor 
approaches a foreign city in the same way. 

Yet there exists neither an all encompassing culture- and arts assistant nor a device to share 
relevant touristic experiences. 


We clarify this cycle based on the simple model of a clock. 


THE E-CULTURE CLOUD 
Subjects 
The E-Culture Cloud covers the following subjects: 


* The need for information per se (many of the aforementioned needs are simply not 
met and if so - not to a satisfactory level) 


* The exchange-of-experience and knowledge (in contrast to conventional offers that 
represent individual or insular interests) 


e The thematic approach (in the sense of associative search and personalized 
exploration) 


e The connection with social media (as a powerful add-on to existing social media 
services) 
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AII needs identified by the Culture Clock should be satisfied by one platform appearing in 
different shapes. 


A search engine / before my trip 

The search for culture takes place not only in advance but literally at all stages of 
experiencing culture. Practically every data based on web-searches is left to commercial 
engines. 

We demonstrate the advantages of a user-centered search-engine. 


A map / during my visit 


Every tourist needs orientation. And maps are not only geographic tools but also serve as 
editorial access to articles, ratings, topics and institutions. 


Augmented Reality-App / during my visit 


AR is not a futuristic vision but working perfectly here and now. We demonstrate how 
maps and search engines interact to create a vivid live-navigation through Florence. 


Indoor-navigation / in the museum 


Based on the iBeacon-tour conceived for Hamburg' s Museum for Commerce and Industry 
we show how indoor navigation adds museal experiences to the cloud. 


A network / following my visit 


Experiences should be shareable in cloud-based networks and the social web. User- 
generated-content enables a recommendation- and association-based search-platform. 


BIG DATA 


Based on these topics we would like to show how cloud generated data can be used to 
benefit cultural institutions and culturally interested people alike. Among other things by 
enabling: 

* atargeted approach to users 

* athematic, associative, connected and recommendation-based search 


e „learning“ technology 


* effective social-media-connection 
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Abstract — In the field of Art and Cultural Heritage, operators are conscious that the public would 
like to be involved more in exhibitions and in the preservation of Culture, so experts and researchers 
are studying new technologies, and, in particular, Virtual Reality, to find new means to engage visitors 
and improve their experience. 

The new headsets and 3D glasses, which allow augmented reality applications, represent one of the 
latest frontiers in the field. 

This paper presents an application of augmented reality, realized through the Epson Moverio and 
Google CardBoard devices, which allow to provide additional information to masterpieces cultural 
sites. 


INTRODUCTION 


New mobile devices and technology has transformed the way to access information, allowing 
people to constantly participate in events feeling involved in determining the success of an 
initiative. Also Museums and, in general, cultural heritage can benefit by the new attitude of people 
to be participative, finding the way to attract and engage visitors in the making of the museum 
experience. Technology can be seen either as a communication medium [9] and as the tools we use 
to solve problems and develop the products that we need (e.g., many museums in United States 
offer workshops and programs that engage children in using technology to design things, make 
tools, and express creativity). For example, current devices and software platforms offer museums 
the opportunity to track visitors and understand what they are doing both in the museum and on- 
line. Moreover, many museums have incorporated surveys or feedback mechanisms into their 
technology devices as well [17]. 

In any case, current technology represents not only the core of many tools we use to solve 
problems and develop the products needed, but it also refers to the medium used to involve people 
in experiencing art. 

The variety of means offered by technology represents an opportunity but at the same time it might 
result in confusing support designers, often introducing tools that are not easy to access or adding 
poor contents if compared to the costs of the implemented solution. Consider, for example, the use 
of Virtual Reality reconstructions, often too expensive or inapplicable for the public because of the 
difficulties in making a Virtual Reality environment accessible at the same time by groups of users 
enabling them to interact with one another and with the VR environment. In this case, the costs are 
not often justified if compared with the results. Moreover, the current 3D technology offers endless 
possibilities of realization: special effects contribute to making Virtual and Augmented 3D Reality 
appealing, thus involving the public. 

In this paper, we present a simple application based on recent low cost technologies, allowing 
museums to involve people through a gaming-based approach and low financial investment on the 
development phase. 
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In the next paragraph we will describe the general state-of-the art, providing an overview of our 
previous researches on which the presented tool is based. In paragraph three we will describe the 
application and in the latter paragraph, we will present our conclusions and further possible 
development. 


RESEARCH, APPLICATION AND TECHNOLOGY TO ENHANCE THE MUSEUM 
EXPERIENCE: AN OVERVIEW 


In this work we present our application allowing museums to offer visitors a simple way to 
experience art when visiting a museum. The application may enable public not only to access 
additional resources on the works of art but also, for example, on the restoration stage of a work, on 
other works of the same style or epoch, on galleries of related works, etc. The presented application 
can have an impact both on the acknowledgement and the valorisation of artistic heritage and 
during its experiencing and sharing. Of course we have to consider that the process of creating 
adequate supports implies great efforts on several different levels: collection of materials and 
information concerning the masterpiece; language translation of materials, taking into account 
different visitors” cultures and behaviours; education purposes, i.e. the implementation of 
methodologies improving long lasting memory to promote the educational aspect of the Art; 
message design and media selection, on the one hand to promote an efficient communication 
process and, on the other hand, to involve visitors in a positive experience both from the cultural 
and the emotional point of view; choice of suitable supports, i.e. from traditional paper versions of 
leaflets and brochures to more advanced technology-based items, all at a reasonably low cost and 
easy to access. The cost of this process has to be considered in terms of expenses but also in term of 
time devoted by experts in producing effective and impactful material. 

Current technology represents the core of many of the above listed activities because it consists of 
the tools we use to solve problems and develop the products needed. 

As has been observed, a very typical phenomenon of our times in Western societies “is the rapid 
transformation of textual traditions and orders of discourse" [8, p.96]. Any change brings about new 
combinations, the co-occurrence of contradictory elements, which give rise to a mixture of styles. 

A few years ago we have started our research, reviewing the state-of-the-art of VR in Cultural 
Heritage [9]. We have then investigated means and methods to collect, analyze and interpret data 
[5] from people accessing art to find a way to enhance the visitors” experience visiting. 

Our researches have been conducted attempting to jointly use methods from technology, education 
and psychology with the aim to enhance the public's experience when observing Art. 

In order to get conscious and unconscious feedback from users, we chose to use EEG-based BCI 
(Brain-Computer Interface) [1]. 

We performed our researches with particular focus on users” emotional and cognitive response to 
musical and visual stimuli, with the aim to transfer our result to enhance users experiencing of art. 
We performed preliminary experiments to evaluate specific protocols with the aim to test the 
reliability of both the Emotiv Epoc and the Neurosky Mindwave. We based the considered mental 
state/emotion labels on the 2D valence/arousal model [12] originating from cognitive theory. This 
model has been used to determine the apparent mood of music in several works [11,13]. 

In further experiments we investigated the exposure to a visual-perceptive, semantic, or conceptual 
stimulus influences response to a later stimulus in the context of a Museum of fine Art [6]. Visual- 
perceptual priming [15,16] is defined by enhanced processing of previously seen visual material, 
relative to novel visual material. The purpose of our research consisted in developing a priming- 
based tool taking into account the most relevant experimental and physiological findings and 
applies them to the museum environment. 

In other works we tested the user reaction to visual stimuli using ambiguous images from Gestalt 
[14], monoscopic and stereoscopic movies [7] and colours [10]. Also in this case, using both 
classical questions-based tests and EEG-signal analysis, we obtained interesting results, confirming 
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the importance of specific colours, shapes and sound, on the one hand to make priming really 
effective and, on the other hand, to have guidelines to use appropriately 3D Virtual Environments to 
engage visitors. We also applied the same concepts and methodology in the context of e-learning 
environments for dyslexic users, with specific focus on learning English as a Second Language [4]. 
Results from this study gave us interesting feedback concerning the appropriate use of general 
communication in different languages, not only for dyslexic users, but also for the ordinary learners 
and suggested important guidelines for the choice of supports and devices that are likely to be more 
appreciated by participants to our experiments. 

All the obtained results have been used to design our prototype that is going to be presented. 


A LOW-COST APP TO ENHANCE MUSEUM VISITORS' EXPERIENCE 


As told, new technologies are widely used in museum for educational, informational, or virtual 
reality simulation aimed at the conservation and protection of cultural heritage and Art. Despite of 
the significant opportunities offered to improve user experience, the costs of these technological 
solutions are often too high and does not justify the expenses in term of return on investment (i.e. 
often the visitors" amount or satisfaction are lower than expected). 

In this paper, we present an application of augmented reality, realized in two versions: through the 
device Epson Moverio and through the Google CardBoard. The app allows to provide additional 
information to the masterpieces exhibited in a museum. 

The first, based on the Epson Moverio device, present the novelty consisting in making "social" the 
experience conducted by the visitors and, in addition, allows to connects the works at the 
“intangible” culture linked historically or in the popular tradition at masterpieces and at artists. In 
this way, the user involvement is reinforced thanks to the gaming approach, to the information 
received and to the social approach of the application. 

Moverio is an augmented reality viewers composed by two parts: the Glasses (the viewers) and the 
Control Unit (where the Operative System (O.S.) is installed). 

The Glasses are structured in two micro-projectors put on the two side of the glass. These devices 
project the image, with a mirror mechanism, on the clear lenses. Currently it is possible to overlay 
an image to reality with a resolution of 960x540 (QHD) and an aspect ratio of 16:9. Furthermore, 
with the presence of the two lenses, one for each eye, Moverio can create an immersive stereoscopic 
image. 

The Control Unit has two functions, one for the touchpad, one for its kernel, where the Android 
O.D. is located. For this reason the application must be developed following Android development 
guidelines. 

The app we created consists of the fruition of multimedia content related to the context where the 
subject is located. Scenes concerning the historical, social, cultural or natural environment are 
recreated in overlay with the external real environment. For example, imagine to stand in front of 
the Colosseum, wearing your Moverio glasses: in front of you some Gladiator and typical 
characters of Roman civilization will appear, just as if you time-travelled to ancient Rome. The 
application can be used both in historical sites (outdoor), using GPS technology, and inside of 
Museums or galleries (indoor), through the use of image recognition technology. 

Due to the cost of Moverio devices, they cannot be distributed to all the visitors of a museum, but 
we can imagine to place one Moverio every three-five masterpieces at the disposition of the public. 
The “social” part of the application consists in a game, allowing users to test their knowledge of the 
masterpieces, the historical periods or the information about an artist. The user can do this game 
alone and after he/she can publish an “award” on his Facebook timeline or participate to a context, 
playing with other visitors and participating, for example, to a context, having his/her name 
published as an "expert", "interested" or “beginner” in the devoted museum page and, if the 
museum provides it, receiving possibly a symbolic prize for the performance (for example a gift, a 
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ticket for an exposition, and so on). Visitors have the possibility to publish the results on their social 
networks, contributing, in this way, to attract also friends in participating in the museum activities. 


$— m 


Figure 1: a schemata of the Moverio app features 


A similar, but simpler, application has been developed for the Google Carboard'. In this case, the 
cost is really low and, even if the application is not interactive as the previous one, it presents many 
advantages, either on the development side and on the attractiveness for visitors. 

The Google Cardboard is a card-made devices that can be bought for a few dollars or can be built 
on your own, just assembling a pair of lenses with a 40mm focal distance to keep the phone's 
screen in focus, a magnet, some velcro, a rubber band, and a tag if you want to tap your device to 
the headset to launch the app right away. The magnet and rubber band serve as a makeshift 
hardware button to your phone. 


Figure 2: the google Cardboard 


A museum can buy a cardboard for 2.5 dollars and so give it to all visitors of an exhibit, allowing 
users to keep the device at home and use it also in the future. The Cardboard allows users to 
visualize, in an immersive 3D environment, any content developed for smartphones, so the museum 
can develop applications in an easy way and allow visitors to download it on their mobile devices 
watching them during the exhibition visit. Here we present an application developed for this aim. 
The application consists in creating an immersive scene, 360-degree wide, were the user can benefit 
from additional multimedia information. We can think, for example, to locate ourself in the 
Gauguin's painting "D'où venons nous/Que sommes nous/Oà allons nous". 

The app allow user to explore a three-dimensional reconstruction of the painting from a focal point, 
for example the center of it, navigating the painting as if the user was inside the scene, walking 
through and looking around exploring the picture as a virtual environment. 


' Official documentations and API to develop applications are provided by Google at 
https://developers.google.com/cardboard/android/ 
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Figure 3: a schemata of the Moverio app features 


For the best use of the immersive features of the device, the use of Cardboard is indicated only for 
indoor locations (like museums, art galleries, etc ..). The effect is, however, truly impressive and 
realistic. 


CONCLUSIONS 


One of the new frontiers in communicating arts and culture is the so called digital storytelling, or 
multimedia tale: the exhibition spaces, museums and galleries may present equipment and 
installations that, through multimedia (words, sounds, lights, photographs, motion pictures, video), 
tell artworks, but also objects, environments, places, aspects of the territory... 

Multimedia tale, however, is a very broad definition, including numerous and different ways of 
using new technologies in the exhibition spaces. The presented technologies, namely the Epson 
Moverio glasses and the Google Cardboard, offer the possibility to use outcomes from previous 
researches, implementing storytelling-based apps helping museums to attract visitors, also 
enhancing their cultural experience. Moverio devices are more expensive, so they are more suitable 
if installed every tree-five masterpiece and can be used by visitors also for interactive game-based 
applications, allowing users to communicate the experience on their own social network pages. 

The Google Cardboards are definitely low-cost, so they can be distributed and left to all the visitors, 
for an additional cost of only two-three dollars to the exhibition ticket. 

With a small economic investment and with low effort in development, we can create value and 
attractiveness. With the presented applications, in fact, the exhibition turns that into a story that 
evolves from the beginning to the end, thanks to moving images, sound, additional information... 
Another advantage consists in the possibility of direct interaction with the work. In traditional 
museums and exhibition spaces, in fact, you have no way to interact with it, you can at most change 
the angle from which you look at it, you can zoom in or out, but nothing more. The use of 
multimedia produces interesting effects on the fruition of the masterpiece. Some researches [2,3] 
confirm that, in general, the use of multiple communication codes - typical multimedia story - 
allows creating a stronger emotional connection between the work and its beholder, that is a better 
feeling. Also on an emotional plane, the narrative-digital dimension facilitates the fruition of the 
object, and this is especially true in the case of a non-expert individual, far from the object itself as 
for interest, curiosity, knowledge. 
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SUMMARY 


The Hamburg Conservatory, in cunjunction with the Hamburg Ministry of Culture, is 
about to establish an eLearning environment as an additional resource for both basic and 
higher education in music (»KON-Plugin«). Junior students receiving elementary training in 
the Conservatory's music school as well as international to-be-professionals enroled in its 
undergraduate and graduate college programs shall soon make extensive use of digital means 
to meet the needs of modern classrooms, be they real or virtual. Hamburg’s eCulture program 
is aiming for an online reflection of the city's cultural assets, providing content, 
comprehensive information and interaction. By adding concepts of eLearning to eCulture, 
music education will, just like other disciplines before, become independent from two or more 
persons being in the same room at the same time, teaching and learning together. This ranks 
from more flexible schedules and mobility within one's own neighbourhood to the sharing of 
curriculums and best practices in Europe and beyond. 


However, turning mere information into usable knowledge and capabilities takes more than 
a broadband Internet connection and suitable Apps. When it comes to music, real teachers are 
essential to make the crucial shift from just absorbing bits and pieces to a learning experience 
that creates sustainable knowledge about art, craft and, most important, other people. An 
eLearning platform must not replace but support the teaching. It is rather the tool than the 
teacher. While implementing eLearning for music education it might be useful to consider: 


* Knowledge can't be spooned to a student. It rather needs to be discovered, led by a 
teacher. 

* Cloud computing on whatever mobile phone or tablet computer is a powerful 
teaching tool, providing easy, time-independent commenting on whatever content — 
audio, video, MIDI, text — that has been uploaded by either the student or the 
teacher. 

* Cloud computing helps to enrich one-on-one real teaching sessions. It is not ment to 
replace them. 

* [n terms of international college ranking meanwhile the quality of community among 
the students outsmarts the quality of the information taught on campus. So, cloud 
computing, on an international level, has to lead to real students travelling to real 
places that have been dealt with before in the virtual classroom. Shared curriculums 
among cooperating institutions could be the framework for this option. 
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TRANSMITTING HYBRID THEATRE & CINEMA ACROSS 
BORDERS WITH LIVE ENGLISH, GERMAN & ITALIAN 
LANGUAGE EVENTS FOR EDUCATION & ENJOYMENT 
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This paper builds on work presented at EVA 2014 Berlin and before with regard to ‘Event Cinema’ 
[or Livecasting and other terms] and new Royal Shakespeare Company [RSC] & Ravensbourne live 
events for schools across the UK. In particular we hope to attract interest at EVA Florence for 
future Italian involvement in this rapidly emerging hybrid art and educational form, originating in 
the Opera field by New York's Metropolitan Opera, the *Met', less than a decade ago. First we 
briefly recount the rise of ‘Event Cinema’ with particular regard to transmitting live Theatre 
performances to cinematic spaces internationally, noting the astonishing success in this field of the 
UK's National Theatre. Second, we describe recent innovative efforts of the Royal Shakespeare 
Company ( RSC) of Stratford-upon-Avon and Ravensbourne [a transformed high-tech London 
Higher Education institution in the post-Olympics re-invented East London] to bring Shakespeare 
productions to 30,000 pupils in classrooms around the UK with expert commentators and theatre 
personalities: ‘live’ with student interaction. Third, we compare and contrast these two approaches 
to identify possible synergies. Fourth, we develop scenarios for corresponding future trans-language 
border initiatives which would for example ideally lead to great Italian Theatre [e.g.by Carlo 
Goldoni and the Nobel prize winner, Dario Fo] being shown ‘live’ in Hamburg & London 
cinema/educational spaces as ‘Special Events” [NB not just streaming to individual computers for 
individual and small family/friends groups]. And vice-versa in Florence for top German and 
English language theatre. Such a Florence/Hamburg &/or Berlin/London initiative could provide 
Proof of Concept for further roll-out in these three countries — especially with regard to educational 
applications as the RSC/Ravensbourne experience shows to be highly promising. Similar trans- 
language border cooperation [e.g. France & Spain] would ideally follow well before 2020 with 
educational and cultural benefits for the future Europe. We conclude by setting this approach within 
the larger picture of the rapidly changing global media landscape, as well as identifying potential 


difficulties and risks such as economic ones, pan-European broad band-width limitations and IPR. 
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Abstract — This paper presents the results of NeoLuoghi, a research project co-funded by the Italian 
Ministry of Education, University and scientific Research (MIUR) under the PON R&C program, with 
special focus on the social networking aspects. The project proposes an innovative methodology as well as 
advanced technologies enabling a new paradigm for fruition of the cultural space and cognitive mediation, 
in *themed-parks", based on the concept of neo-places (neoluoghi) as opposed to Marc Augé's non-places. 
It also studies how to leverage Social Network analysis (social mining) to profile users, understand their 
interests and offer them engaging interactions with the cultural asset. 


INTRODUCTION 


Themed-parks can be envisioned as neo-places of cultural experience (narrative and sympathetic 
museums, equipped urban areas or real experiential themed parks) representing the fully participated 
metaphor of the relationship between society and cultural values, a new world of knowledge experience 
and aesthetic shared emotion. The project strategy is to consider cultural neo-places such as themed 
systems with a high density of information, which require integrated solutions to evoke, perform, 
transmit, manage and customize projective and evocative messages. 


Archetypes , Territory: tangible Network, other media, other 
and intangible heritages > storytelling 


KIW \ lee 


Transmedia storytelling: » full Information supply 
experience customization - myths and figures 

+ online services 

* multimedia storytelling 
Social network: » thematic analysis 
experience sharing + digital artifacts created 
by users 


* Myths and figures 
* Localized services 
* Paths and storytelling 


Physical Park Virtual Park 


Figure 1: Communicating a cultural park 
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The project objective is to define methodologies and tools to operate at multiple levels on the neo- 
places of cultural experience, to: (i) understand, raise, design and create content; (ii) provide 
infrastructure and advanced services of "ambient intelligent" and (iii) make them sustainable and 
functional to strategies for territorial development. 

To demonstrate the method, the project is developing innovative applications based on emerging 
mobile and wearable devices allowing end users to experience the cultural neo-place accessing related 
information and content, included multimedia and augmented reality. The visiting experience is 
complemented by social networking, allowing users to share content and experiences in a collaborative 
or competitive effort (e.g. geo social games, prizes competitions). 

The social interactions and connectivity during the visit of the themed park, and in general during 
content fruition, is implemented through the NeoLuoghi Social Network (NL-SN), whose members are 
the visitors of the themed park. The NL-SN is a tourist geo-localized SN (similar to 
Foursquare/Swarm) that allows the user to access mechanisms of social interaction, during the 
storytelling based visit. The NL-SN provides recommendations on points of interest (POI), access to 
competitive and collaborative discovery games, ability to post photos and comments and to discover 
visitors with similar interests whom to meet during the experience. This is accomplished by profiling 
the visitor and also on the basis of the user's SN aspects and behavior. To profile the user, after 
requiring proper authorization, data on user activities on SN (SN data) is collected and analyzed (social 
mining), thus obtaining a more complete user profile. 

The key challenge in the design of the social interactions is to give users the benefit of social 
interactions (typically, specialized recommendations) while maintaining an unobtrusive and immersive 
user experience where the focus is on the visit (e.g the monument or art work) and the technology is 
just a precious invisible support. 

NeoLuoghi innovative method allows to realistically enforce the business assumption of systematic, 
efficient and flexible market coverage of the national landscape of cities and places of culture, starting 
from the the city of Naples and Campania region, where the project is being validated, addressing 
suggestive venues like ancient Roman's Paestum and Gianbattista Vico's Baroque Naples. 

In the next section, we give an overview of the general objectives of the project as well as the 
methodology developed in NeoLuoghi for the automated production of myth-related narrative scripts 
for themed parks in a scenario of cultural fruition. In the main section, we will present the results of the 
projects concerning the visitor experience and in particular, the management of the social networking 
interactions in the themed park. Finally we propose some indications for further work. 


METHODOLOGY 


The project considers five major aspects, namely (i) the perception of cultural systems, both 
historicized and recognized, both potential and achievable through initiatives of cultural engineering; 
(ii) the visual, spatial and functional architecture of the experiential systems; (iii) the experiential 
systems and ambient intelligence (internet of things, internet of everything); (iv) experiential marketing 
of services for the enhancement of territories and cultural resources (social marketing); new 
development strategies for cultural experiential systems. 

Mythopoesis is a narrative genre in modern literature and film where a fictional mythology is 
created by the writer of prose or other fiction. The authors in this genre integrate traditional 
mythological themes and archetypes into fiction. Narratology refers to both the theory and the study of 
narrative and narrative structure and the ways that these affect our perception. 
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The project NeoLuoghi aims at rationalizing and modeling the process of elaboration of the 
"experiential script" of a themed place, while keeping the process the most repeatable and reusable in 
new contexts. If, as seems evident, a themed place qualifies and becomes operational for its 
imaginative efficiency, i.e. its ability to immerse users in a fabula conducted personally by the visitor / 
customer / tourist is of the utmost importance to be able to identify and effectively to evoke a pervasive 
and imaginative world, which identifies the Park (implicitly or explicitly) and the experience that is 
able to transfer to those who visit and live it. 

The project developed a methodology for the analysis (detection, classification, valuation) of the 
pervasive themes that run through contemporary society, as opportunities for thematic representation 
and identification. Furthermore, for each of the possible driving themes, the project provided a 
methodology for the narratological development of theme itself, such as: in a specific imaginative 
context, what needs to be said and lived, in what order and with what stylistic features, with which 
relationship between the self and the environment, in what structures congenial to elective socializing. 

The project produced an attempt of methodological classification and “industrialization” of the 
creative processes that are implicit in the activities of architectural workshops during the development 
of themed and experiential entertainment systems. 

The aim is to rethink the processes of visual ideation and spatial composition on the one hand, and 
of synthesis between architectural and exhibition set up on the other. As of today, such processes are 
customized, intuitive, creative, and “cultural” in the deepest sense. The aim is to extrapolate, in the 
narrow domain of interest of the project, the constants, rules, strategies, conduct, reference models, and 
ultimately a set of organized knowledge that can effectively support the work of those who design, 
enrich and organizes themed urban spaces and immersive theme parks . 

In detail, the project provides: (1) the definition of the imagery in the Park, namely the salient 
elements of style and visual image, of symbolic and descriptive scenery that translate the deep script of 
the themed space in its organization and visual communication; (ii) the study of “spatial narrative” of 
the park, in the relationship between the plain narrative of a cultural story-myth and the ways in which 
a system of aesthetic suggestions spatially unfolds: his proxemics, that is the logical organization of the 
spatial dimension in which the user interacts, the fundamental architecture of the space, and then in 
theatrical dramatization of space; (iii) the study of models of construction and urban design: the 
selection and implementation of urban and didactic apparatus (the “implicit parks”, i.e. the historic 
urban centers), and the creation of architectural artefacts and the set up of exhibition spaces in real 
theme parks. 


SOCIAL NETWORK ASPECTS OF THE USER EXPERIENCE 


One of the most challenging objectives of the project NeoLuoghi is to deliver an immersive yet 
unobtrusive user experience, leveraging emerging information and communication technologies. The 
main device considered for content fruition is a tablet where a native application smartly displays the 
required content types, provided by web applications as well as stored locally. To this aim the first 
aspect considered is the modulation of information in line with the users and their characteristics, 
through profiling techniques and solutions for dynamic narrative content generation (storytelling), 
consistent with the mission of the experiential park. The second aspect concerns the strengthening of 
the experiences that visitors can enjoy in the park, either implicitly or explicitly, thanks to new 
techniques for creating multiple points of interest in the urban fabric and in the park, solutions to 
manage social and emotional interactions enabling a cooperative and engaging experience of the park, 
and augmented reality solutions to create a more immersive and emotional viewing experience. 


120 


The social interactions and connectivity during the visit of the themed park, and in general during 
content fruition, is implemented through the NeoLuoghi Social Network (NL-SN), whose members are 
the visitors of the themed park. The NL-SN is a tourist geo-localized SN (similar to Foursquare) that 
allows the user, during the narrative development of the visit, to access mechanisms of social 
interaction. The NL-SN provides recommendations on points of interest (POI), access to competitive 
and collaborative discovery games, ability to post photos and comments and to discover visitors with 
similar interests whom to meet during the experience. 

The NL-SN ability to recommend content and POI is accomplished by profiling the visitor on the 
basis of: (i) her behavior during the visit, (11) visitor's social interactions in the NL-SN, and (iii) the 
present and past behaviors of the visitor in external SN (e.g. Foursquare/Swarm, Twitter, TripAdvisor). 
To profile the user, data on user activities on SN (SN data) is collected, after requiring proper 
authorization, by either directly the NL-SN system or extracted by crawling third parties SN. External 
SN data points are retrieved either through their native Application Programming Interfaces (API) or 
through SN aggregators, such as Gnip, LoginRadius or Hootsuite). 
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Figure 2: Social mining 


Since SN data typically have different formats and different meanings, it is normalized and then 
associated to the user profile in order to complete it with SN aspects. Subsequently the complete user 
profile is used as input to recommendation algorithms for the definition of content and POI 
recommendations. A part of the user profile will be characterized by social aspects, both originated 
from external sources (other SN, social sections of external non social websites, e.g. Amazon) and 
originated by the NL-SN. Additional information completes the description of the user profile, in 
particular the dynamic data collected by the fruition application on the user's tablet, such as 
geographical location, interest shown, preferences, pathways, interaction with the NeoLuoghi's themed 
park POI 

In addition, the system develops simple recommendations concerning the activities of the user on 
the SN (social recommendations) and provide them to the user. In particular, the recommender system 
provide visitors (1) advise on people geographically close or mission-related to meet and share the park 
experience and (ii) recommendation about common activities of interest, such as the presence of events 
or games in the area . 

Finally, the system collects recommendations originated by external SN and forward them to the 
user device where they are displayed properly (e.g. using non-intrusive side scrollable list) trying to 
associate them to the POI / location visited . 

The key challenge in the design of the social interactions is to give users the benefit of social 
interactions (typically, specialized recommendations) while maintaining an unobtrusive and immersive 
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user experience where the focus is on the visit (e.g the monument or art work) and the technology is 
just a precious invisible support. 
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Figure 3: Part of the NeoLuoghi system architecture and SN module. 


SOCIAL MINING AND RECOMMENDATIONS: AN IMPLEMENTATION 


We used MongoDB as content repository, for its speed in data storage and support for JSON format, 
a Javascript-based open standard format! used to exchange data among distributed web applications 
and services, widely used by SN APIs and SN aggregators. 

MongoDB is a document-oriented NoSQL-Database. MongoDB stores records not in tables as a 
relational database but in BSON documents, which is a binary version of JSON and very similar to the 
object structure. The usage of MongoDB makes his development easier and deployment faster. 

In addition, we used ElasticSearch to query MongoDB, since it is very efficient for query large and 
complex databases, again with JSON support. 

Elasticsearch lends itself well to a variety of interesting ways to process the vast amount of content 
in social media. Many recommendation engines use Apache Mahout’, we preferred ElasticSearch for a 


1 JSON (Java Script Object Notation) uses human-readable text to transmit data objects consisting of attribute 
value pairs. It is used exchange to transmit data between a server and web application, as an alternative to XML 
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more general support to query data, while Mahout is specialized on user profile data (e.g. Mahout is 
capable of recommending articles based on user's preference). In addition, ElasticSearch is dedicated 
and optimized for data query, and it is a perfect complement for MongoDB. 

Elasticsearch is a Java-based search server based on Lucene. It provides a distributed, multitenant- 
capable full-text search engine with a RESTful web interface and schema-free JSON documents. It 
provides scalable search and near real-time search. 

Elasticsearch supports data-intensive distributed applications and implements a computational 
paradigm named MapReduce (via the plugin Taste from Grouplens) The Taste plugin for 
ElasticSearch is Mahout Taste-based Collaborative Filtering implementation, providing the following 
features: (1) Data management for Users/Items/Preferences, (ii) Item-based Recommender, (iii) User- 
based Recommender, (iv) Similar Users/Contents and (v) Text Analysis. 


CONCLUSIONS AND FURTHER WORK 


This paper presents the results of the social networking aspects of the NeoLuoghi project, a large 
research initiative addressing a new paradigm of fruition of the cultural space and cognitive mediation 
in “themed-parks”. The project proposes an innovative methodology as well as advanced technologies 
for modeling the process of elaboration of the “experiential script" of a themed place, while keeping 
the process the most repeatable and reusable in new contexts. 

The first part of the research was dedicated to investigate the mechanisms by which the 
contemporary sensibility perceive cultural systems. In addition, key enabling technologies are adopted 
to transform themed parks into "smart" places in which information is handled and distributed 
efficiently, improving visitor experience and enjoyment. 

Subsequently, we designed a system architecture integrating the different system components and 
implemented the user interface of the tablet application for content fruition. Finally, the system is 
currently being evaluated with real users, in real cultural scenarios in Naples. 
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